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Executive Summary 


The Maryland State Department of Education (MSDE) is leading and supporting the implementation of a 
Teacher and Principal Evaluation (TPE) system in all school districts in the state. This study examines the 
progress of four years of TPE implementation, 2013 to 2016." 


The study draws on nearly 60,000 survey responses from principals and teachers (1,905 responses in 
2013, 16,314 in 2014, 19,022 in 2015, and 21,916 in 2016) and extensive interviews of educators at 
central and school site levels. Based on the ongoing analysis of these data, the key findings are as 
follows: 


Overall four-year trends 


TPE implementation shows strong progress over four years 


Based on four years of data, educators’ perceptions of TPE are improving as they gain experience 
implementing all components of the evaluation system. This trend has held true over all four years of the 
study on TPE. 


e The increases in positive perceptions of principals and teachers on TPE implementation are 
significant. 


e There is constant year-to-year growth in educators’ positive perceptions of TPE on all measures 
tracked during the four years of implementation. 


e Principals and teachers indicate that TPE has clear expectations, leads to improved 
instructional decisions, and is better supported with each year of implementation. 


e Principals and teachers also emphasize that the instructional focus of TPE is making a 
significant difference in schools. 


State support for TPE implementation is pivotal 


MSDE is taking a building block strategy to developing educators’ understanding of and ability to 
implement TPE. The impact of MSDE’s strategy is reflected in constant year-to-year improvements in 
educators’ perceptions during the four years of TPE implementation. 


Salient elements of MSDE’s sophisticated implementation effort are to: 


e Create a statewide model for TPE, while communicating a vision that TPE should support 
continuous improvement of instruction. 

e Convene and help sponsor extensive statewide and regional training sessions for teams from all 
24 districts, other key district constituencies, and more than 85% of Maryland’s institutions of 
higher education. 

e Coalesce the state’s eight major educational entities through a Memorandum of Understanding? 
(MOU) to garner widespread support for quality implementation of Student Learning Objectives 
(SLOs). The MOU received national attention as an innovative example of educator engagement. 


e Effect a collaboration of MSDE and the Maryland State Education Association (MSEA) to support 
district and school leaders. 


e =Train multiple cohorts of promising principals, with participants from every district. 


e Engage external partners to study educator perceptions about TPE, and actively use findings to 
inform TPE strategies in response to educator opinions. 
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School and district leadership is critical to TPE implementation 

Districts have discretion to implement TPE to fit their local context. Districts vary, in the amount and 
types of professional development and support they provide to principals, teachers, and central 
administrators; in the level of involvement of teachers; and in their focus on implementing TPE to 
improve instruction rather than to comply with state requirements. There are marked differences in the 
approaches districts take to support TPE implementation. 


e The districts that are making real progress in the consistency of quality implementation are 
doing so through concerted efforts to build principal and teacher capacity and to align TPE with 
the instructional framework and priorities of the district. These include districts that are 
advanced in their implementation of TPE as well as districts that are making progress after initial 
slow starts. 

e There is asmall subset of districts that have lower levels of educator agreement on the quality 
and impact of TPE implementation. These districts tend not to take advantage of state 
sponsored trainings, or they fail to take what is learned at those trainings and apply it in the 
district. Some districts take shortcuts in an effort to make the process more manageable, which 
often results in undercutting the power of TPE to improve instruction. 


Principals and teachers still need support 
Principals and teachers are developing a more nuanced understanding of what is required to implement 
TPE at a high level of effectiveness. They are learning from their experience. 


e Principals and teachers agree that school and district support for the implementation of TPE is 
helpful. 

e Principals and teachers want more support in all areas of TPE to increase their instructional and 
managerial effectiveness. 

e Most districts are increasingly aware that the needs for greater capacity have to be addressed. 


Quality, consistency, and manageability 


Quality matters 

Quality matters when implementing SLOs and teacher observations. The quality of the capacity building, 
the quality of dialogue related to meeting student academic needs, and the quality of the planning and 
delivery of instruction all matter. When SLOs and observations are implemented with fidelity in 
Maryland’s districts, and backed by the full support of district leaders, there is increased focus and 
reflection on instruction and student progress. 


Maryland districts with strong implementation: 


e Develop their SLO process with the input of teachers in the district, and provide ongoing 
professional development that helps teachers use SLOs to improve their classroom practice. 


e Have a pre-observation conference, the observation of the teacher in the classroom, and a post- 
observation conference using the same evaluator for all three tasks. 


e Engage teachers in dialogue with their evaluator about instructional practice. 
e Use results to make continuous improvements in instruction. 


Consistency is a cornerstone of effective TPE implementation 


Achieving consistency of implementation at a high level of quality is an ongoing challenge within and 
across districts and schools. It is a key issue that affects the reach and credibility of the evaluation 
system. 
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Districts that are building a consistent, quality implementation across schools are doing so through 
strategic efforts. Their leaders make a concerted effort to: 


e Build a consistency of understanding among educators at central and school site levels so that 
the district has a shared view of TPE implementation. 


e Use rubrics to provide both a standard and a basis for supporting the quality of SLOs and the 
quality of observations. 


e Engage in formal training and team sessions to develop consistent approaches and inter-rater 
reliability for SLOs and classroom observations. 


Districts need thoughtful management of the evaluation system 


The leaders who are effective in implementing TPE make it a district-wide priority to thoughtfully support 
and manage the evaluation system. 


Key requirements for making TPE manageable are to: 


e Set clear expectations for TPE to improve instruction. 
e Train principals and teachers on high quality implementation strategies. 


e Create a meaningful principal evaluation system to better understand and address the needs of 
principals. 


e Create technology platforms and information management systems that support TPE 
implementation and enable districts to identify recurring challenges across multiple classrooms 
or schools. 


Transition to the Every Student Succeeds Act (ESSA) 


In the transition to ESSA, Maryland's educators want to stay the course 


District and school leaders are characteristically making the same statement—they want the evaluation 
system to continue to focus on instructional improvement. Many districts have been working to change 
their practices and cultures. There are a number of other districts that are now moving in these 
directions. On a recurrent basis, many leaders and practitioners are underscoring two key points—stay 
the course in the focus and support of TPE, while allowing more flexibility in the weighting of the 
components. 


Issues and recommendations 


The recommendations focus on strengthening leadership and further improving implementation at 
central and school site levels within Maryland’s districts. 


/ssue One: Statewide implementation of change 


Recommended action 
e Base the ESSA evaluation strategy on TPE’s progress. 
e Renew the eight organization alliance in support of evaluation and instructional improvement. 
e Use the learnings from TPE implementation to inform new statewide initiatives. 


Impact 


Maryland is a leader in implementing new educator evaluation systems. Maryland’s success to-date is 
based on substance and process. It should shape future TPE implementation under ESSA and inform 
other statewide reform initiatives. 
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/ssue Two: District support capacity 


Recommended action 
e Provide advanced training to school site and district level practitioners. 
e Focus on instructional improvement to increase accountability. 
e Emphasize teacher voice and constituency building. 
e Engage the local boards of education. 
e Conduct a thoughtful reset in select districts. 


Impact 

Nurturing a new direction for instructional improvement and evaluation requires leaders who can change 
practices, mindsets, and the culture of central offices and schools. Maryland’s progress to-date provides 
the foundation for the steps which need to be taken to deepen the impact and reach of TPE. 


/ssue Three: Principal leadership 


Recommended Action 
e Increase the capacity of principals to guide and oversee implementation. 
e Make principal evaluation a greater priority in districts. 


Impact 

The credibility and usefulness of a new principal evaluation system lies in whether it recognizes and 
provides the vehicle for improving performance. Principals of all performance levels need an evaluation 
system that produces these outcomes. 


Summary 


Maryland is making significant progress in implementing the new teacher and principal evaluation 
systems, as reflected in survey and interview data over a four-year period. Moving forward, district- and 
school-based educators indicate that Maryland should stay the course in the focus and support of TPE, 
while allowing more flexibility in the weighting of the components. This means continuing to link TPE to 
instructional improvement and continuing to strengthen the quality, consistency, and manageability of 
implementation. 
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Chapter One: Introduction and Context 


Purpose of this study 


Since 2013, the Maryland State Department of Education (MSDE) has led and supported the 
implementation of a new Teacher and Principal Evaluation (TPE) system throughout the state. For each 
year of implementation, MSDE has partnered with the Mid-Atlantic Comprehensive Center at WestEd 
(MACC@WestEd) to examine the implementation of TPE. 


The MACC@WestEd study and technical assistance team in Maryland, a collaboration of WestEd and 
the Community Training and Assistance Center (CTAC), has previously released three comprehensive 
reports documenting the progress of TPE implementation in Maryland: Spotlight on Maryland: Student 
Learning Objectives and Teacher and Principal Evaluation (2013), Real Progress in Maryland: Student 
Learning Objectives and Teacher and Principal Evaluation (2014), and Change in Practice in Maryland: 
Student Learning Objectives and Teacher and Principal Evaluation (2015). MSDE has used the findings 
and recommendations from the reports to inform the state’s support to educators throughout Maryland. 


This report documents the progress and current state of TPE and how it has developed over the last four 
years. The study examines the perceptions of educators in Maryland of the support they receive in 
understanding and implementing the TPE system. Findings of this study are intended to help further 
inform and strengthen the implementation of the TPE system in Maryland. 


Background of this study 


TPE began with a pilot in seven districts in the 2011-2012 school year. In 2012-2013, a field test was 
conducted in all schools within 22 school districts, all of which signed on to implement Maryland’s Race 
to the Top grant. The first full statewide implementation of TPE took place in 2013-2014. Each district 
constructed an evaluation model based on its interests and priorities, and each local superintendent and 
head of the local bargaining unit signed off on the design. MSDE’s ESEA flexibility waiver request, 
approved by the U.S. Department of Education in July 2014, recognizes school year 2014-2015 as the 
baseline year for student achievement. Under the waiver, Maryland will not use state assessment scores 
in personnel decisions during the first two years of the Partnership for Assessment of Readiness for 
College and Career (PARCC) assessments. 


Table I-1. TPE implementation overview 


School Type of TPE 


Year Implementation Rarecinaliie 


e 7 districts: Baltimore City, Baltimore County, Charles County, Kent County, 
2011-2012 TPE Pilot Prince George’s County, Queen Anne’s County, St. Mary’s County 
e 83 schools, 934 teachers, 48 principals 


e All schools within 22 districts, all of which signed on to implement Maryland’s 
Race to the Top grant 
3 districts implemented the full state model 

e 19 districts developed their own local models approved by the state 


2012-2013 Field Test 


9013-2016 | Full Statewide | | au schools within 24 districts 
Implementation 
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Maryland's TPE system 


The statewide TPE system in Maryland includes measures of both professional practice and student 
growth.’ In the overall rating of teacher effectiveness, both professional practice and student growth are 
given equal weight. 


The professional practice portion for teachers includes four domains outlined by the Charlotte Danielson 
Framework for Teaching (i.e., planning and preparation, instructional practices, classroom environment, 
and professional responsibilities). The four domains are further broken down into 22 components. Each 
of the four domains accounts for 12.5% of the professional practice section, totaling 50% of the total 
evaluation score. For principals, the professional practice measure consists of eight domains based on 
the Maryland Instructional Leadership Framework and four domains from the Interstate School Leaders 
and Licensure Consortium. The 12 domains are weighted individually to reflect the needs of principals at 
varying times in their careers. 


Student Learning Objectives (SLOs) are the predominate measure of student growth for teachers and 
principals. The state TPE models allot 30%-50% of the total evaluation rating to SLOs, depending on the 
assignment of the teacher and principal. No single SLO, however, may count for more than 35% of the 
total performance score. (See Figures 1 and 2 in the appendix for the state’s graphics that illustrate 
these models.) 


District variations 


The historical and current practice in Maryland is one of local control. The key implementation decisions 
and the organizational supports that reinforce those decisions are made at the district level. The local 
school systems in Maryland have the discretion to choose to develop local plans and include additional 
measures that align to their local priorities. These local plans are required to include certain criteria and 
measures defined by the TPE framework that every evaluation model must have (e.g., the 50/50 split 
between professional practice and student growth, the four Danielson-like domains for teachers, and the 
eight Maryland Instructional Leadership Framework domains for principals). In addition, local models 
need to have the endorsement of the local collective bargaining unit and must receive approval from the 
state. Most districts follow the state model for professional practice with some variation in the weight 
given to sub-measures of professional practice. All districts use SLOs, but the number and weighting in 
the overall teacher effectiveness rating of SLOs varies. Districts have different ways of combining the 
two scores for professional practice and student growth to create an overall rating of teacher 
effectiveness.® 


MSDE key state strategies 


MSDE has taken a proactive role in leading the implementation of TPE and purposefully utilizing a broad 
range of strategies to continuously and strategically address the emerging needs of stakeholders. All of 
these efforts demonstrate MSDE’s commitment to identifying the changing needs in the field and making 
mid-course improvements. MSDE made it a departmental priority to provide districts with the 
organizational and financial resources needed to support successful implementation. 


Key MSDE strategies have included: (a) engaging key stakeholders by coalescing and signing a 
landmark statewide Memorandum of Understanding (MOU) between the Maryland State Board of 
Education, the Maryland State Department of Education, the Maryland State Education Association, the 
Public School Superintendents Association of Maryland, the Maryland Association of Boards of 
Education, the Maryland Association of Secondary School Principals, the Maryland Association of 
Elementary School Principals, and the Baltimore Teachers Union; (b) supporting the Promising Principals 
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Academy and collaborating with the institutions of higher education to build the leadership corps needed 
to sustain the work; (c) providing ongoing professional development (e.g., statewide and regional SLO 
trainings); (d) promoting open and two-way communications (e.g., the TPE Communication Bulletins; 
meetings); and (e) learning from external researchers and experts. 


In 2015-2016, MSDE was proactive in building district capacity for high quality TPE implementation. The 
state TPE team worked with the MOU signatories, superintendents, an executive staff advisory group, 
and a principal advisory group to help them strengthen the process of implementation in all districts. 
MSDE also worked with district research and teachers’ association representatives to review evaluation 
models and processes to help local teams better understand the effectiveness of their TPE rating 
models. In addition, MSDE and MSEA collaborated in sponsoring TPE training sessions held in the fall, 
winter and spring for district teams of executive staff, principals, and teachers, as well as for teacher and 
administrator preparation programs. These sessions focused on strengthening SLOs, observations, 
assessments, and equity. The district teams also received assistance in how to use action planning so 
they could promote fidelity of TPE implementation within and across schools. 


MSEA support under the MOU 


In support of the MOU, the Maryland State Education Association (MSEA) received a grant from the 
National Education Association to build the capacity of educators to implement TPE. MSEA also created 
a cadre of teacher member leaders (i.e., Student Learning Objectives Leadership Cadre, or SLOLC) from 
around the state to deliver professional development on SLOs and build local capacity. In July 2014, 
MSEA sponsored leadership training in which cadre members were provided two full days of content 
specific support on SLOs. The MSEA SLOLC members also participated in the statewide training 
sessions co-sponsored by MSDE in the 2014-2015 school year. Cadre members became resources 
supporting or leading professional development in several districts. MSEA provided additional support 
and resources to cadre members throughout the year. 


Every Student Succeeds Act (ESSA) 


On December 10, 2015, the Every Student Succeeds Act (ESSA), was signed into law. ESSA signifies 
another major development in the broader context of TPE. Under ESSA, states are no longer required to 
set up teacher evaluation systems based in significant part on students’ test scores. However, states are 
still required to adopt “challenging” academic standards.’ 


In Maryland, MSDE is working with the ESSA Stakeholder Committee to review the new law and develop 
Maryland’s ESSA Plan. The ESSA Stakeholder Committee is charged with providing guidance on the 
transition to ESSA; and providing recommendations for the State Superintendent of Schools and the 
State Board of Education on Maryland’s ESSA Plan. 
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Chapter Two: Methods of Data Collection and Analysis 


For this descriptive study, a mixed methods approach is used with both quantitative data and qualitative 
data collected from educators across the state. The multiple sources of data include statewide survey 
responses provided by teachers, principals, and other educators from all 24 school districts, phone 
interviews of educators from six districts and the state level education association, and site visits to 
three districts. 


Maryland teacher and principal evaluation survey 


A web-based confidential survey, developed by the study team, was launched on March 1, 2016. This 
was the fourth annual administration of surveys by the study team. The Maryland State Department of 
Education (MSDE) sent an invitational email with the link to the survey to all superintendents in the state, 
asking them to share this information with the principals, teachers, and other school-based staff in the 
local school districts and encourage them to respond. The study team alone had access to survey 
responses. MSDE updated the superintendents regularly on the progress of survey participation. The 
survey was open seven weeks until April 22, 2016. 


The multiple choice questions centered on: (a) the systemic context for the implementation of the TPE 
system; (b) the quality of the TPE frameworks and processes; (c) classroom observations; (d) supports 
teachers receive on Student Learning Objectives (SLOs); (e) educators’ experience with the TPE system 
during the 2015-2016 school year; and (f) additional supports educators need. An open-ended question 
was included which provided an opportunity for the respondents to share any comments they had 
concerning the TPE system. 


The statewide response rate to the TPE survey has increased over the years. It was 31.3% in 2014 with 
16,314 responses, and 30.3% in 2015 with 19,022 responses. For the 2016 TPE survey, originally 22,072 
educators responded to the survey. A total of 156 surveys are excluded from the study for not identifying 
their district affiliations. The final analysis is based on the responses of 21,916 educators from all of the 
24 local school districts. The district response rates range from 10.2% to 85.3%, with a state average of 
34.6%. 


The majority of the respondents are classroom teachers (68%), and nearly two-thirds of the teachers 
have 10 or more years of teaching experience (67%). Most of the survey respondents have a Master’s 
degree (76%). A total of 713 principals completed the survey, constituting 49.3% of all school principals 
in the state, as well as 522 assistant principals. The respondents to the 2016 TPE survey are very similar 
to those to the 2014 and 2015 surveys based on such observable characteristics as TPE experience, 
school level, position, subject area, experience as a teacher, experience as a principal, and highest 
educational degree (see Table 2 in the appendix for more detail on the characteristics of the survey 
respondents). 


The survey analysis includes an examination of the responses by three teacher categories: classroom 
teacher, non-classroom teaching staff, and special educator. Since teachers’ responses were very 
similar across their categories, they have been grouped into one category, “Teacher,” for the purposes 
of discussion in the report. For similar reasons, the principal and assistant principal responses have 
been grouped into one category, “Principal.” Cross tabulations and figures were used to describe 
educators’ perceptions. Mann-Whitney U tests were conducted to examine the statistical significance of 
the differences across groups and years. This study’s analysis is based on responses from teachers and 
principals as described above. 
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Figure Il-1. Number of survey responses, aggregate and by position, 2014-2016 


25,000 
21,916 
20,155 
20,000 19,022 
17,344 
16,314 
14,905 

15,000 
10,000 
5,000 

1,029 1,160 1,235 

; — 2 

State Total Principals Teachers Other 


m2014 ©2015 2016 


Note: Principals refers to principals and assistant principals; 7eachers refers to classroom teachers, non-classroom teaching staff, 
and special educators; and Otherrefers to other administrators, central office staff, and those who did not report their position. 


Key leader interviews 


Confidential phone interviews, using interview protocols developed by the study team, were conducted 
from late March to mid-June 2016 with 15 key stakeholders in six districts and one statewide 
association. Districts selected for stakeholder interviews were based on district size geographic 
representation throughout the state. Each interview was approximately one hour long. 


The six districts vary by district enrollment, the percentage of ethnic minority students, and the numbers 
of schools, teachers, and administrators. Comparisons based on these observable characteristics 
suggest that the six districts constitute a diverse cross-section of all of the districts in the state. 


Site visits to three districts 


In-depth site visits were conducted in three districts in early 2016. The site visit in each of the three 
districts consisted of individual interviews and focus groups. Districts selected for site visits had high, 
medium, and low levels of agreement, respectively, among teachers and principals in the previous year’s 
survey results. The purpose of the site visits was to add context to the survey findings. Each individual 
interview was approximately one hour long. 


The study team conducted focus groups within each district with, respectively, teachers and principals. 
Districts were asked to select 8-10 participants for each focus group, making sure the teacher and 
principal focus groups had representatives from elementary, middle, and high schools. For the teacher 
focus groups, the study team also requested a mix of teachers in tested and non-tested grades and 
subject areas. Each focus group was approximately ninety minutes long. All interviews and focus groups 
were conducted using protocols developed by the study team. 
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Table Il-1. Surveys and interviews, 2016 


Method of 


Data Collection Details/Participants 


e Launched March 1, 2016, closed April 22, 2016 


aun e Statewide response rate: 34.6% with 21,916 responses 


e Phone interviews with: 

5 superintendents 

5 local teacher association leaders 

1 statewide teacher association leader 
1 teacher association representative 

1 central office administrator 

1 principal 

1 teacher 


Key Leader Interviews 


o0o0o0060dUCOlOUUlCO 


e Individual interviews and focus groups 
e Total of 50 participants: 

o 2 superintendents 
3 local teacher association leaders 
7 central office administrators 
16 principals (in focus groups) 
22 teachers (in focus groups) 


Site Visits to Three Districts 


1@) 
1@) 
[e) 
10) 


Limitations of the data 


In order to gain actionable information and timely insights to further inform the implementation of the 
TPE system, data were collected and studied within a relatively short time frame. Although the study 
team analyzed educators’ survey responses from all 24 districts in the state, the response rates and 
number of respondents by district vary considerably. Meanwhile, the data are perceptual in nature. No 
student performance data were available to the study team for analysis. 


In addition, it is important to acknowledge that the survey data are not strictly longitudinal because the 
surveys across the years have been anonymous. It is therefore not possible to identify whether the same 
teachers and principals are responding from one year to the next. Consequently, although the 
characteristics of the respondents have been highly comparable across the years (i.e., Appendix Table 
2), it is still possible that some of the changes observed can be functions of the different samples in 
different years. 


Furthermore, although the nine districts for the phone interviews and site visits cover a broad range of 
district characteristics in terms of enrollment and the numbers of schools, teachers, and administrators, 
the findings in those districts may not be generalizable to all of the districts. 


Despite these limitations, consistent findings and trends across the years emerged from the survey 
responses, interviews, and focus groups. Together they draw a clear picture of what respondents see as 
the strengths and areas of improvement of TPE, SLOs, and classroom observations. These findings have 
important policy and practice implications for the continued implementation and improvement of teacher 
and principal evaluation. 
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Chapter Three: TPE Implementation 


Four years tell the story of TPE implementation in Maryland. Principals and teachers are primarily 
positive in describing their overall perceptions of TPE and the type of progress that is being made. They 
also have perspectives on what is needed to further strengthen the quality, consistency, and 
manageability of implementation. 


Among the key findings: 
e TPE implementation shows strong progress over four years. 
e Improvements continue in 2016. 
e Instructional focus makes the difference in schools. 
e School and district support for TPE is critical to effective implementation. 
e Principals and teachers still need support. 


TPE implementation shows strong progress over four years 


With each year of implementation, principals and teachers are increasingly positive about the 
evaluation system. This trend in survey responses has held 
true over all four years of the study on TPE. 


Educators’ perceptions of TPE are improving as they gain MSDE is taking a building 
experience implementing all components of the evaluation 
system. Table Ill-1 shows the percentage of educators who 
agree with the seven statements that have been used in all 


block strategy to developing 
educators’ understanding of 


four surveys from 2013 to 2016. For every survey item, there and ability to implement TPE. 
has been a year-by-year increase in the percent of principals The data indicate that this 
and teachers who agree with the statements on “Systemic approach is proving distinctly 
Context,” the “Quality of TPE Frameworks and Processes,” successful. 


and “Student Learning Objectives.” 


These responses show a pattern that continues through many 
survey items. Principals and teachers are both increasingly positive with each year of implementation 
and principals’ levels of agreement are generally higher than those of teachers. 


Table Ill-1. More educators agree TPE has clear expectations, leads to improved instructional 
decisions, and is better supported, 2013-2016 


Principals Teachers 


2013-16 5913 2014 2015 2016 2219-16 


Systemic Context 2013 2014 2015 2016 Grange Ghance 


The expectations are clear for teachers 


9 9 9 * * 
and principals in the TPE system. 55% 70% 82% 88% +33%* 41% 50% 63% 68% +27% 


Quality of TPE Frameworks and Processes 
Teacher evaluation frameworks and processes... 


Lead to improved decisions related to 


9 9 9 * * 
instructional approaches. 66% 79% 82% 83% +17%* 38% 48% 53% 58% +20% 
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Student Learning Objectives (SLOs) 
Teachers receive... 


Information about developing high 
quality SLOs. 

Guidance in selecting learning content 
for SLOs. 

Guidance in selecting research-based 
instructional strategies for SLOs. 


Recommendations for selecting pre/post 
assessments for SLOs. 


Valid and reliable approaches to setting 
growth targets for SLOs. 


2013 


57% 


50% 


45% 


44% 


42% 


2014 


70% 


76% 


63% 


71% 


60% 


Principals 


2015 


76% 


79% 


70% 


15% 


65% 


2016 


85% 


87% 


76% 


79% 


17% 


2013-16 
Change 


+28%* 


+37%* 


+31%* 


+35%* 


+35%* 


2013 


42% 


32% 


27% 


29% 


28% 


2014 


50% 


50% 


37% 


48% 


40% 


Teachers 


2015 


57% 


56% 


45% 


53% 


47% 


2016 


64% 


63% 


52% 


57% 


54% 


2013-16 
Change 


+22%* 
+31%* 
+25%* 
+28%* 


+26%* 


Note: Throughout this report, Principals refers to principals and assistant principals; 7eachers refers to classroom teachers, non- 
classroom teaching staff, and special educators; Agree is a composite of strongly agree/agree; and Disagree is a composite of 
strongly disagree/disagree. * indicates statistically significant differences at the 95% confidence level. N (principal, 2013 = 173; 
2014 = 1,029; 2015 = 1,160; 2016 = 1,235); N (teacher, 2013 = 1,657; 2014 = 14,905; 2015 = 17,344; 2016 = 20,155). 


MSDE is taking a building block strategy to developing educators’ understanding of and ability to 
implement TPE. The data indicate that this approach is proving distinctly successful. Table Ill-1 shows 
the impact of MSDE’s strategy: there is constant year-to-year growth in educators’ positive 


perceptions of TPE on all measures tracked during the four years of implementation. 


Moreover, as Figure Ill-1 shows, the increase in the percent of respondents who agree with these 
statements over the full four-year period —comparing 2013 responses to 2016 responses —are 
significant. The increases in principals’ positive perceptions of TPE implementation range from 17 to 37 


percentage points over four years. 


Figure IIl-1. Changes in principals’ perceptions of TPE, 2013 compared to 2016 
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As Figure IIl-2 shows, teachers have similar growth in positive perceptions of TPE implementation from 


2013 to 2016. The increases in teachers’ positive perceptions of TPE implementation range from 20 to 


31 percentage points over four years. 


Figure IIl-2. Changes in teachers’ perceptions of TPE, 2013 compared to 2016 
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Improvements continue in 2016 


Reflecting MSDE’s emphasis on continuous improvement, a deeper look at survey responses from 2015 


to 2016, the most recent two-year comparison, shows that changes in practice and changes in 


perception are both positive and continuing during the period under study. Table III-2 shows that 
principals and teachers have higher levels of agreement on nearly all factors in the survey related to 


implementation. As educators gain experience with the TPE system, and the districts and schools start 
to more systematically address challenges related to implementation, educators are more likely to have 


positive perceptions about the system and its implementation. This is consistent with findings from the 


prior year’s study of TPE.® 


As shown in the “Systemic Context” section, both principals and teachers show an increase in positive 
perceptions, from 2015 to 2016, of the expectations of the TPE system and of the support they receive. 
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Table IIl-2. More educators agree on TPE progress, 2015-2016 


Principals Teachers 
Systemic Context 2015 2016 Change 2015 2016 Change 
The expectations are clear for teachers and principals in the 82% 88% 46%" 63% 68%  +5%" 
TPE system. 
My district s support of the implementation of the TPE 81% 88%  47%* 46% 57% +411%* 
system is helpful. 
My school’s support of the implementation of the TPE 88% 91%  +4+3%* 57% 63%  +6%* 


system is helpful. 


Quality of TPE Frameworks and Processes 
Teacher evaluation frameworks and processes... 


Provide useful feedback to teachers. 87% 89%  +2%* 54% 62%  +8%* 
Encourage reflection on instructional practices. 90% 89% -1% 70% 72%  +2%* 
Lead to improved decisions related to instructional 82% 83%  41%* 53% 58%  +5%* 
approaches. 


This Year’s Experience 
This year, | believe... 


There were opportunities to confer with my principal about 88% 90%  42%* 67% 68% 41%" 


my SLOs. 
There were opportunities to review and refine my SLOs. 79% 91% +12%* 64% 75% = +11%* 
Reflection on my instructional practices deepened. 71% 77% ~~ +6%* 50% 54%  +4%* 


There was more instructionally focused dialogue with my 75% 79%  +44%* 48% 5A% 469%" 


colleagues. 

There was more instructionally focused dialogue with my 78% 81% 43%* 44% 48% 44%" 
supervisor. . : : : : 
The results of my evaluation will be used to inform my 75% 76%  41%* 49% 51% 42%" 


professional development plan for next year. 
Note: * indicates statistically significant differences at the 95% confidence level. 


Instructional focus makes the difference in schools 
Both principals and teachers increasingly indicate that 


TPE processes provide useful feedback to teachers 
and lead to improved instructional decisions. 


The intent of the TPE system is for 
districts to own the model and use tt to 


The intent of the TPE system is for districts to own the improve instructional practice, not just 
model and use it to improve instructional practice, not comply with the state requirements. 
just comply with the state requirements. The i 
responses in “This Year’s Experience” (Table IIl-2) 
show that both principals and teachers are more likely 
to agree in 2016 than in the previous year that there are opportunities to confer with the principal, to 
refine SLOs, to reflect on practice, to have instructionally focused dialogue, and to base professional 
development on TPE. These responses all relate directly and positively to TPE’s intended instructional 
focus. 


Stay the Course: Teacher and Principal Evaluation in Maryland 14 


In interviews, educators indicate that TPE is helping improve classroom practices. 


“There is more substantive discussion on what is happening in the classroom. With TPE, 
principals and teachers are really looking at student progress. They focus on instruction more than 
they used to.” 

-Superintendent 


“There is a very strong relationship in the model between TPE and instruction. It gets teachers to 
dig down and focus on instruction and what good instruction looks like. I do think there is a 
strong relationship where we break the framework down to components. It allows teachers to show 
what they do and discussions between administrators and teachers as a process developed.” 


-Union Leader 


“What TPE is doing for teachers is its impact on instruction. It forces me to look at the strategies I 
use in the classroom. I have to look closely and I am looking at the new standards. The part that is 
key to success of the process is that once we establish our target, we focus on how to achieve that to 
the best of our students’ capabilities and that leads to differentiated strategies.” 


-Teacher 


School and district support for TPE is critical to effective implementation 


Increasingly more principals and teachers agree that school and district support for the 
implementation of TPE is helpful. As Figure III-3 demonstrates, a preponderance of principals agree 
that district support of TPE implementation is helpful (88%) and even more agree that school level 
support is helpful (91%). The majority of teachers have favorable perceptions regarding district support 
(57%) and school support (63%) of TPE implementation, though not at the same level as the principals’ 
responses. 


Figure IIl-3. Both principals and teachers see district and school support of TPE as helpful, 2014-2016 
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Note: Throughout this report, Agree is a composite of strongly agree/agree and Disagree is a composite of strongly disagree/disagree. 
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In interviews and focus groups, educators indicate that their perceptions of the merits and overall 
implementation of TPE is a function of the level and type of support educators receive. For most 
educators, that support comes from the district and the school, while many have also participated in 
state and/or association trainings sponsored by MSDE and MSEA. 


There are districts that have approached supporting implementation strategically from the very start. 
There are also some districts which started more slowly, but then made TPE a greater institutional 
priority and reset the process of implementation. Yet there are still several districts that need to take a 
hard look at their current implementation efforts. 


At key junctures, some districts recognize the need to revisit their TPE system and see what needs to 
change to be more effective. Several districts re-cast their approach to SLOs to better support 
instructional improvements in the classroom and to increase the levels of teacher and principal 
buy-in. Others realize that teacher and principal turnover means that districts must provide the same 
level of training and professional development to new hires as it originally offered to staff in past years. 
Many districts are also seeing that the TPE process requires ongoing improvement in the quality, 
consistency, and manageability of implementation. 


“We had to rebuild the processes that just were not there. We had a really collaborative 
arrangement with the association. We were in a position to have conversations on how to make it 
better, not just checking boxes, so we could have a thoughtful process around how we drive the 


work,” 
-Superintendent 


“I believe that the TPE system has made great strides toward improvement since last school year. 
T appreciate the effort that the administrators (county and school) have made to make the 
evaluations more meaningful and have more substance and clarity.” 


-Teacher 


“My SLOs went very well this year due to the enormous support from my principal, AP, and 
teammates. We did use data to guide the SLOs and monitor progress. We also tried to focus on 
borderline students.” 


-Teacher 
“The process seemed much more seamless and reflective this year.” 


-Teacher 


Many district leaders indicate that TPE has brought significant changes to their school systems. They 
cite a number of positive changes including, specifically, the focus on instruction and on student 
academic growth: 


“There have been significant signs that we are moving from the checklist mentality to growth 
mentality and using rubrics gives us a nice chance. It also helps to have more goal setting — it’s 
leading to more reflections. That's really helpful. On the student growth side...it has caused people 
to be more specific in their goal setting and look at student data in a different way.” 


-Superintendent 
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“Getting specific to the evaluation, I think we have learned that in general, we do a better job than 
we did before because we have more instructional conversations with teachers, and we have data to 
talk about it.” 

-Superintendent 


“I feel there has been, in many schools, a more collaborative coaching style approach to the 
evaluation process.” 
-Union Leader 


Principals and teachers still need support 


Principals and teachers show in their survey responses that they are developing a more nuanced 
understanding of what is required to implement TPE at a high level of effectiveness. Simply put, they are 
learning from their experience. In this context, principals and teachers are now identifying the areas 
in which they need more support. 


Table IIl-3 shows a number of areas where principals and teachers agree they need more support. These 
needs are discussed more fully in Chapter 5. 


Table Ill-3. Principal and teacher levels of agreement on capacity and support, 2016 


Capacity Building 
To increase my effectiveness as a teacher or a principal, 
there needs to be more support at my school in... 


Principals Teachers 
(n=1,235) (n=20,155) 


Analyzing student data for action. 1% 49% 
Using assessments to measure growth. 70% 45% 
Using Maryland's College and Career-Ready Standards. 65% 44% 
Linking SLOs to Maryland’s College and Career-Ready Standards. 64% 43% 
Receiving SLO feedback from school administrators (for teachers only). N/A 46% 
Receiving SLO feedback from district administrators (for principals only). 57% N/A 
Using data from SLOs and observations to strengthen school improvement 69% 47% 
planning. 
Using data from SLOs and observations to make improvements in my instructional 

: N/A 49% 
practices (for teachers only). 
Using data from SLOs and observations to make improvements in my instructional 69% N/A 


supervision (for principals only). 


As educators’ knowledge about the TPE system increases, they are becoming more aware of the 
additional capacities they need to strengthen their practice. Most districts are increasingly aware that 
the needs for greater capacity have to be addressed. 


“If you evaluate me but do not see where my needs are, and nothing is done about it, eventually the 


process will be of no value.” 
-Superintendent 


Stay the Course: Teacher and Principal Evaluation in Maryland 17 


“While our district has conducted numerous trainings on approvable SLOs, more work needs to be 
done in building teacher capacity to use data to drive instruction as well as the best research-based 
instructional practices.” 

-Principal 


“I'd like more meetings about good instructional strategies. I'd like to hear from colleagues about 
what is being done in their classrooms. I'd like to participate in teacher-driven walkthroughs that 
foster meaningful collegial conversations about instructional practice.” 


-Teacher 


Summary 


With each year of TPE implementation, more principals and teachers are positive about TPE. In addition, 
as educators increase their knowledge about and comfort with the TPE system, they are becoming more 
aware of where they need to strengthen their capacities. Many districts are providing more support to 
help educators increase their effectiveness with all aspects of the TPE system. Interviewees indicate that 
there are also districts that are struggling with TPE implementation. Where the TPE system is not being 
implemented well, there is value for leaders in re-examining their district’s approach to TPE and 
conducting a thoughtful re-start with renewed training and intensive support. Across all districts, the 
priorities of district leaders play a significant role in determining the quality, consistency, and 
manageability of TPE implementation. These issues are examined in Chapters 4, 5, and 6. 
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Chapter Four: Quality 


Quality matters. When implementing SLOs and teacher observations, the quality of the capacity building, 
the quality of dialogue related to meeting student academic needs, and the quality of the planning and 
delivery of instruction all matter. 


Quality of SLO implementation 


When SLOs are implemented with fidelity in Maryland’s districts, and backed by the full support of 
district leaders, there is increased focus and reflection on instruction and student progress. The 
challenge for districts and schools is to continue to improve the quality of implementation so that both 
teachers and students benefit. This is an evidence-based approach as research shows that high quality 
SLOs are positively associated with higher student achievement.? 


MSDE has made concerted efforts to bolster district understanding of SLOs and to increase the quality 
of SLO implementation. Survey data show that these efforts are paying off. Continuing a four-year trend, 
principals and teachers in 2016, compared to 2015, increasingly agree that teachers are receiving more 
support on all aspects of implementing SLOs. More than half of the teachers and nearly three-quarters 
of principals agree. 


As the survey data in Table IV-1 shows, there continue to be increases in the percentage of principals 
who agree that teachers are receiving information, guidance, and recommendations on the use of SLOs 
in classrooms. Principals have higher levels of agreement in 2016 compared to 2015 on the support 
teachers receive, ranging from 71% of principals who believe teachers receive information about 
connecting Maryland's College and Career-Ready Standards to SLOs, to 85% of principals who agree 
that teachers receive information about developing high quality SLOs. 


Teachers have higher levels of agreement in 2016 than they had in 2015 about the support strategies 
they receive for SLOs. Teachers’ agreement on the support they receive ranges from 51% who believe 
they receive information about connecting Maryland's College and Career-Ready Standards to SLOs, to 
64% who agree that they receive information about developing high quality SLOs. Across the range of 
SLO support strategies, teachers’ levels of agreement continue to increase, although principals’ levels of 
agreement remain higher. 


Table IV-1. More educators agree on SLO support for teachers, 2015-2016 


Student Learning Objectives (SLOs) Principals Teachers 

Teachers receive... 2015 2016 Change 2015 2016 Change 
Information about developing high quality SLOs. 76% 85% +9%* 57% 64% +7%* 
Guidance in selecting learning content for SLOs. 79% 87% +8%* 56% 63% +7%* 
Guidance in selecting research-based instructional 70% 76% 46%" 45% 52% 47%" 
strategies for SLOs. 
Information on connecting Maryland's College and - 0 wie 3 0 ape 
Career-Ready Standards to SLOs. cial iis ue noe ate Oe 
er ale for selecting pre/post assessments 75% 79% 44%" 53% 57% +4%* 


Valid and reliable approaches to setting growth targets 65% 77%  412%* 47% 5A% 47%" 
for SLOs. 
Professional development that is informed by the SLO 71% 73% 42%" 49% 53% 44%" 
process. 

Note: * indicates statistically significant differences at the 95% confidence level. 
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Educators indicate that the power of SLOs lies in their ability to focus teacher attention on students who 
need more instructional support. 


“I think the SLO is a powerful piece. If you really have the opportunity to say to a teacher, “your 
group of children lags behind students at other schools and I want you to focus on that group,’ it 
can be used in that powerful way.” 


-Superintendent 


“SLOs force teachers to justify why they chose specific measures. In the conversations with the 
principal or the assistant principal, they have to justify why they chose the measure for their 
students and how it would lead students to success.” 

-Teacher 


“The whole process has totally changed the way we talk at school. It allows us to be more focused, 
but it also gives us more of an idea of where we need to go from where we are. It allows us to align 
what we are doing. If they come in and observe me and they know my SLOs, they are able to see 
the evidence that our SLOs are improving. They talk about it at our conference, and they talk to 
our students about what our goals are. What I really like is they talk about students when they 
come in. The discussions on students then transfer back to the classroom and parents.” 


-Teacher 


Districts and SLO support strategies 


There are marked differences in the approaches districts take to support SLO implementation. Districts 
vary, in the amount and types of professional development and support they provide to educators; in the 
level of involvement of teachers; in their approach of using teacher-developed versus centrally-defined 
SLOs; and in their focus on using SLOs to improve instruction versus using SLOs simply to obtain a 
teacher rating. 


Ongoing support and professional development for SLOs 


One way to measure the relative strength of the SLO implementation process is to determine the 
percentage of teachers who receive SLO support in seven key areas, as identified in Table IV-1: 


e Information about developing high quality SLOs. 

e Guidance in selecting learning content for SLOs. 

e Guidance in selecting research-based instructional strategies for SLOs. 

e Information on connecting Maryland's College and Career-Ready Standards to SLOs. 
e Recommendations for selecting pre/post assessments for SLOs. 

e Valid and reliable approaches to setting growth targets for SLOs. 

e Professional development that is informed by the SLO process. 


The survey responses on these items make it possible to determine the percentage of teachers who 
agree that they have received at least five of these forms of SLO support in their district. Results are 
aggregated to the district to show how many teachers in each district received at least five SLO support 
strategies. Districts are then rank ordered and aggregated by quartile. 
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Figure IV-1 shows the results for each of the four quartiles of districts on the percentage of teachers who 
receive at least five SLO support strategies. In the top quartile of districts with stronger SLO 
implementation, 62% of teachers agree that they receive at least five SLO support strategies. At the 
other extreme, in districts with weaker SLO implementation, only 42% of teachers agree that they 
receive at least five SLO support strategies. This difference of 20 percentage points means that some 
districts are better preparing their educators for successful results than other districts. 


Figure IV-1. Teacher perceptions on SLO support, grouped by district quartiles, 2016 


Percentage of teachers receiving five or more forms of SLO support 


1st Quartile of Districts 62% 


0% 10% 20% 30% 40% 50% 60% 70% 


Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to SLO 
support strategies. Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other 
graphs in this report. 


Districts with stronger implementation provide extensive and ongoing professional development 
on all aspects of SLOs, including: how to judge SLO quality; how to select a target population; how to 
identify learning content linked to standards and research-based instructional strategies; how to select 
assessments and set growth targets; and how to use the SLO process to identify teacher professional 
development needs. Districts with weaker implementation short circuit the teacher and principal thinking 
process that makes these steps meaningful. 


One of the key tasks for district and school leaders is to help 
teachers understand what high quality means in an SLO, and how 


it differs from low quality. Training sessions sponsored by MSDE and Districts with stronger 
MSEA have emphasized the importance of annotating high and low implementation provide 
quality SLOs so teachers and principals can learn what comprises extensive and ongoing 
better practices. A number of leaders have replicated these training professional development 


sessions in their own districts. In these districts, educators indicate on all aspects of SLOs. 
that they are making progress in determining and improving the 
quality of the SLOs. 
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“Our administration team will have principals bring SLOs, put them on screen, and have a 
discussion right there. That helps to balance what is working, what is not working, what makes it 
a strong SLO, and getting principals on board. If we get our principals on board, they deliver it 


back to schools.” 
-Superintendent 


“Our annotations were done by us — what was wrong and what worked. We had to meet across 


subject areas and critique each other’s SLO.” 
-Teacher 


“We had a lot of SLOs that we pulled apart. We examined what was good, what was bad.” 


-Central Administrator 


One area where teachers and principals need additional professional development is in selecting growth 
targets in their SLOs. 


“SLO targets are often set arbitrarily and more guidance on appropriate growth targets based on 
historical data or statistical formulas would make that process less wishy washy.” 


-Teacher 


“As an administrator, I highly value the opportunity to discuss in detail the instructional 
strategies and implications with the teachers as we review an observation or talk about the SLO 
data. The concern is that teachers want to exceed on the SLO and so selecting the goal of an SLO 
and establishing the criteria is a struggle. More guidance and support is needed to help teachers 
set goals that are stretches for them as well as their students.” 


-Principal 


“I think one area [in which] teachers really struggle is target setting...We have had the 
information, but I just think that’s an area that is still confusing.” 


-Union Leader 


Engaging teachers in meaningful collaboration 

Districts with stronger SLO implementation share a number of strategies in common. They develop 
the district’s SLO process with the input and support of teachers (such as engaging teachers as true 
partners on the task forces that developed the process) and they review progress regularly and 
collaboratively in order to make continued improvements. 


“We worked really hard to build relationships because I am a firm believer that if we are in the same 
boat, we need to row in the same direction. We don’t need to agree with the district on everything, 


but if we are not talking to each other, we are not going to make progress.” 


-Union Leader 


“We share SLOs and see what other schools are doing, helping them to come up with more rigorous 


SLOs.” 
-Superintendent 
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“I think we do still have some challenges, but I think we developed a really good process between 
district and association — the key players come together and talk through issues at the table, 
hammering out every issue we may have. We have a process where we are not ignoring each other 
or not talking to each other about issues. We have the right people at the table to address issues.” 


-Union Leader 


“We have a strong TPE system that leverages the partnership between central office administrators 
and teacher leaders. Our SLO work has been consistent and constantly focused on student 
learning. We need to support our teachers in using data to improve instruction.” 


-Principal 


Teacher-developed SLOs and instructional dialogue 

Districts with stronger SLO implementation emphasize teacher-developed SLOs. Drawing from 
lessons learned from the national and statewide research on SLOs,"° they emphasize the importance of 
the dialogue about student needs between teacher and principal, and provide teachers with the 
opportunity to reflect on the learning content, instructional strategies, and their impact on improving 
student performance. 


“It has made people talk about outcomes. Whether they like it or don’t like it, the conversations, in 
my view, were not happening in the state before SLOs were brought in. I think it is a very positive 


outcome.” 
-Superintendent 


“There are now conversations which were not normally taking place. Those are cool conversations 


across multiple content areas.” 
-Principal 


“It’s much more instructional. If you have fidelity to the evaluation system, you have to have 


instructional conversations.” 
-Superintendent 


“SLOs have been a great way to open up a dialogue with others about instructional techniques and 


reflection.” 
-Teacher 


As the school staff increase the focus on instructional dialogue, interviewees indicate that the culture in 
the schools begins to change. 


“The culture is changing. There are more conversations formally and informally on instruction tied 
to our common instructional language. I guess I would say since we adopted this as a coaching 
model, the SLO process makes teachers know they are responsible for student achievement. 
Teachers and principals have conversations outside the formal process. Those are positive 


attributes.” 
-Superintendent 
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“In some schools, at least in our school, we start to look at a data wall, tracking things. We spend a 
lot of time doing data reflections, checking SLOs to see how we are doing. A lot of schools 
incorporated that and the principal discussed with teachers their concerns and their successes. It 
has opened more conversations and you are not in isolation anymore.” 


-Union Leader 


“It’s mandated. There is no argument. So let’s make it meaningful. We are beyond the functionality 
of SLOs. It took us three years. For teachers in the content areas, they are in. They get SLOs and 
how it works. We want it to be a year-long expectation. Our effective teachers all say they saw 
value in SLOs. ‘Hey, what is the best way to raise your student performance and what is your 
evidence to support that?’” 

-Principal 


Opportunities to confer and refine SLOs 


Where districts encourage teachers to have ownership of SLOs, teachers have the opportunity to 
engage in dialogue with their principal about their SLO, and teachers have the opportunity to revise their 
SLO where necessary based on the analysis of data. 


In survey responses, teachers indicate if they agree that they have opportunities to confer with their 
principal about their SLOs, and to review and refine their SLOs. Results are aggregated at the district 
level to show how many teachers in each district had both opportunities. Districts are then rank ordered, 
and aggregated by quartile. 


Figure IV-2 shows the results for each of the four quartiles of districts on the percentage of teachers who 
confer with their principal about SLOs and have the opportunity to refine SLOs. In the top quartile of 
districts with stronger SLO implementation, 82% of teachers agree that they have both opportunities. At 
the other end, in districts with weaker SLO implementation, only 56% of teachers have both opportunities. 
This difference of 26 percentage points means that critical parts of the SLO process are valued highly 
in some districts and less so in other districts. This difference affects the quality of implementation. 


Figure IV-2. Teachers’ opportunities to confer about and refine SLOs, grouped by district quartiles, 2016 


Percentage of teachers who have opportunities to confer with 
their principal about SLOs and refine their SLOs 


1st Quartile of Districts 


3rd Quartile of Districts 


Ath Quartile of Districts 
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Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to SLOs. 
Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other graphs in this report. 
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Focus on improved instruction 

Ultimately, the districts with stronger implementation recognize that the SLO process is about 
improving instruction. The focus is on student learning and professional growth. The teacher ratings 
are a helpful output of the process, but not the central driving force of the process. 


“We have deliberately tried to focus on the student learning piece. We talked about student 
engagement and student response much more so than teacher behavior. I see good growth.” 


-Superintendent 


“You have to recognize what is happening, you have to go back to the question: are kids learning 
and how do you know? If you know that and can write it into an SLO and quantify it, you can 


map out the growth strategies.” 
-Superintendent 


“It takes strong leadership to keep people from going through the motions. I have one principal who 
told me that really good teachers don’t even need to plan. I said, ‘Really? Tell me who your best 
teachers are in the building.’ That principal later said, ‘Well, they do plan.’ It’s not about filling 
out forms or checking boxes. It is about taking information you have about what your kids know 
and are able to do, and incorporating that, matching it with content, and helping them to be 
engaged.” 


-Superintendent 


“I do love the SLOs. It helps to keep a school grounded and focused on goals.” 


-Superintendent 


Where districts have been able to show teachers how the SLO process supports the broader 
instructional goals of the district, educators report they can better make sense of how to use SLOs to 


improve their classroom practice. 


“We recognize that we need to make sure that our 

eee teachers know the relationship between the content 
The districts with stronger 

asian y e they are teaching and what we want students to learn. 
ee He The curriculum work has to be the driver. We are still 
that t] re SLO pros working to align what is taught with what the 

about improving instruction. standards are and what the assessments cover.” 


-Central Administrator 


“Lam diligently completing the SLO work and there is value in the process. It gives me the 
opportunity to reflect on teaching practices and student performance.” 


-Teacher 


“The logical connection to SLOs is the College and Career-Ready Standards.” 


-Principal 
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Districts with weaker SLO implementation fall short in one or more areas. They provide inadequate 
professional development. They sometimes centrally mandate the content of SLOs regardless of the 
student needs in individual classrooms. They tend not to involve teachers in the process in substantive 
ways beyond contractual obligation. Or they focus on the SLO as a compliance activity. The net effect of 
poor implementation is lack of teacher buy-in and knowledge about SLOs, poorly developed SLOs, and 
a focus more on the rating or score produced by the SLO than on instructional improvement. 


“At our school, we have no choice in selecting learning content and setting growth targets for 
SLOs. Our principal tells us exactly what content area and growth targets we're using for the 
SLO!" 

-Teacher 


“We end up focusing on compliance and not on meaningful reflection on teaching and learning.” 


-Principal 


“The system seems well-designed but poorly implemented. It isn't used to inform instruction in 
my building and seems to just be extra work and not useful to me or the school.” 


-Teacher 


“There was very little support to develop and refine the SLO. Administrators were not there to 
guide new staff through the process.” 
-Teacher 


Quality of classroom observations 


Teacher observations are a central component in determining the quality of instruction at the schools. 
They also comprise 50% of the overall teacher ratings in Maryland. Districts have different models or 
protocols they use in conducting observations. Most districts aspire to implement the most commonly 
accepted practice of a pre-observation conference of the teacher and the evaluator, the actual 
observation of the teacher in the classroom by the same evaluator, and a post-observation conference 
of the teacher and that same evaluator. 


In the 2015 TPE report,'' Maryland’s educators indicated that actual observation practice often differed 
from best practice. They cited the many demands on the time of principals in all districts. They also 
noted the significant inattention to best practices in some districts. This year’s survey included items 
intended to increase the understanding of how observations are being implemented in districts. 


Survey responses show that 98% of principals and 81% of teachers agree that observations are 
conducted by observers/evaluators qualified to do the evaluation. But teachers are also more likely to 
agree that some steps in the observation process are skipped—and in interviews they cite the absence 
of the pre-observation conference or the post-observation conference. 


Survey responses show differences in how districts are approaching the observations. In the survey, 
teachers indicate if they agree that their classroom observations include a pre-observation conference, 
the observation, and a post-observation conference, and have the same observer/evaluator conducting 
the conferences and the observations. Results are aggregated to the district to show how many 
teachers in each district have both opportunities. Districts are then rank ordered, and aggregated by 
quartile. 
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Figure IV-3 shows the results for each of the four quartiles of districts on the percentage of teachers 
whose classroom observations are accompanied by pre- and post-observation conferences from the 
same evaluator. In the top quartile of districts with stronger classroom observation implementation, 79% 
of teachers agree that they have all of these levels of support. At the other extreme, in districts with 
weaker classroom observation implementation, only 32% of teachers have such support. This disparity 
in responses of 47 percentage points means that the quality of the observation process in some 
districts is significantly stronger than in others. In some districts, the observations are approached 
seriously. In others, the observations are essentially a compliance activity. 


Figure IV-3. Teacher perceptions on the observation process, grouped by district quartiles, 2016 


Percentage of teachers who receive observations with pre- and 
post-observation conferences from the same evaluator 


3rd Quartile of Districts 
4th Quartile of Districts 


0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 


Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to 
observations. Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other graphs 
in this report. 


Perceptions of the observation process 

In the districts where the observations are highly valued, leaders emphasize the importance of ensuring 
quality in the process. These districts also provide ongoing professional development to teachers and 
principals. 


“I think the principals are making very good strides on achieving a high level of observation 
quality... The superintendent plans to keep making that stride...Most of our principals are well 
educated on that and moving along in a good way.” 

-Union Leader 


“We are constantly building the capacity of all administrators in the central office and the schools 
to ensure we have everything consistently applied across the board. If not, we have to go in and 
provide personalized professional development. I asked schools to work rigorously on teacher 
observations and the evaluation process.” 

-Superintendent 


“Having consistent messages is really important. We have to continue to push that conversation 


on observations.” 
-Superintendent 
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“We are becoming better. The teachers are supposed to reference the rubric. For each subcategory, 
you have to rate it. Our principal is very focused on what the Danielson framework says. We self- 
rate, and I would say our teachers and principals tend to rate things the same.” 


-Teacher 


Educators also highlight the importance of being purposeful in emphasizing instructional improvement 
throughout the observation process. 


“You do observations to see where the teacher can grow. Are we focusing on the areas where a 
teacher is not strong? How does that help me to help that teacher?” 


-Principal 


“I was fortunate to be in a building where my administrator will say, ‘I see your weaknesses, very 
specific items, hey, I will come back again, and I want to see XYZ." That’s exactly what should be 
occurring, but it is not occurring in all buildings.” 

-Union Leader 


“I think the instructional conversation really happens in the post-observation conference. If a 
teacher walks out of the conference and if there is not a constructive instructional discussion, 
something goes very wrong.” 

-Principal 


Summary 


Quality matters when implementing SLOs and teacher observations. The quality of the capacity building, 
the quality of dialogue related to meeting student academic needs, and the quality of the planning and 
delivery of instruction all matter. What matters for quality is the type and strength of district support for 
an instructionally focused TPE system. Districts with strong SLO implementation develop their SLO 
process with the input of teachers. They support implementation by providing ongoing professional 
development that helps teachers use SLOs to improve their classroom practice. In districts with a 
strong, quality observation process, teachers understand what is expected and engage in dialogue with 
their evaluator about instructional practice. Lack of meaningful district support for either the SLOs or the 
observations diminishes their power in shaping the dialogue and thought process necessary for 
continuous improvement of instruction. 
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Chapter Five: Consistency 


Consistency is a cornerstone of effective TPE implementation. Achieving consistency of implementation 
at a high level of quality is an ongoing challenge within and across districts and schools. It is a key issue 
that affects the reach and credibility of the evaluation system. 


Four years of experience in Maryland show that the districts making real progress in the consistency 
of quality implementation are doing so through concerted efforts to build principal and teacher 
capacity. They are intentionally working to bring more consistency to the TPE process through a 
number of inter-related efforts. They are focusing on instructional improvement, while working to 
improve the district and school support to school-based educators. These districts have also been 
active participants in the training sessions sponsored by MSDE and MSEA. 


In districts where TPE is being implemented with less consistency, teachers are quick to point out how 
things are being done differently from building to building, and within buildings. These districts are often 
perceived by the teachers as having leaders who give inadequate attention to what happens with TPE 
implementation at the schools, place less priority on sharing and taking action related to better practices 
identified by the state, and tend to participate less in the sponsored training sessions. 


Leadership commitment to consistency 


With all the competing demands on districts, superintendents underscore the importance of being 
focused—and staying focused—on the consistent implementation of TPE. The district leadership 
role is paramount to successful TPE implementation. 


“Iam working hard to make sure we don’t lose consistency. I want to make sure we have that 
continuity.” 
-Superintendent 


“For principals, we have a meeting once a month. The whole meeting focuses on an instructional 
practice that is important to them. In the beginning of the year, we focus on the strategies and the 
process to ensure we have continuity with the learning strategies from the training.” 


-Superintendent 


“We spent time monthly going through all the training together, rating different classrooms. Then 
we looked at that monthly to make sure we were doing things the same way. Maybe because of the 
other priorities, lately we have not been doing that as much. But our consistency comes from the 
supervisors who are working across buildings.” 

-Superintendent 


Consistency across districts 


There are meaningful differences in the consistency of TPE implementation across Maryland’s 
districts. Survey responses highlight these differences, as exemplified by teacher responses to the item, 
“My district’s support of the implementation of the TPE system is helpful.” In Figure V-1, a grouping of 
Maryland’s 24 school districts by quartile shows a difference of 20 percentage points in responses 
between the top quartile (where 68% of teachers agree that district support is helpful) and the lowest 
quartile of districts (where only 48% of teachers agree). What this means is that educators in some 
districts feel they are being provided with helpful support, while their peers in other districts 
indicate they are not getting the help they need. 
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Figure V-1. Teacher views on helpfulness of district support, grouped by district quartiles, 2016 


Percentage of teachers agreeing district's support is helpful 


1st Quartile of Districts 68% 


2nd Quartile of Districts 60% 


3rd Quartile of Districts 55% 


Ath Quartile of Districts 48% 


0% 10% 20% 30% 40% 50% 60% 70% 80% 


Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to district support. 
Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other graphs in this report. 


Differences within and between districts 

The helpfulness of school level support varies considerably by district. Teacher responses to the 
item, “My school’s support of the implementation of the TPE system is helpful,” show an 18 percentage 
point difference in response rates between the top quartile of districts (where 73% of teachers agree that 
school support is helpful) and the lowest quartile of districts (where only 55% of teachers agree). These 


variations have meaning because the more teachers are effectively supported, the greater their 
chances for success. 


Figure V-2. Teacher views on helpfulness of school support, grouped by district quartiles, 2016 


Percentage of teachers agreeing school's support is helpful 


1st Quartile of Districts 


2nd Quartile of Districts 


3rd Quartile of Districts 


4th Quartile of Districts 


0% 10% 20% 30% 40% 50% 60% 70% 80% 


Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to school support. 
Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other graphs in this report. 
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Differences within and between schools 


A major concern for teachers is when they perceive differences in how the evaluation system is being 
implemented from teacher to teacher within a school. 


“My conversations with the principal are not the same as other teachers... was brought into the 
system and I was not trained like everyone else. I had to do a lot of learning on the fly. That 
inconsistency is a problem.” 

-Teacher 


“Inconsistency within disciplines...That is a real issue and concern.” 


-Teacher 


In different schools, teachers cite inconsistencies in how various components of TPE are being 
implemented. Some teachers discuss how the observation process is being conducted, while others 
note variation in how SLOs are rated. 


“The observation process is highly subjective and expectations are applied inconsistently. 
Conflicting observations between a supervisor and principal are not resolved equitably. I’m 
speaking as someone who was part of the evaluation construction process in my county.” 


-Teacher 


“The SLO portion of the evaluation system is too inconsistent in both process and significance. 
Importance placed upon SLOs varies from administrator to administrator and school to school. 
This inconsistency sends the message that SLOs are not a viable tool for evaluation.” 


-Teacher 


“I have found that administrators in my school implement the standards inconsistently, some not 
understanding the nuance of the framework and the classroom experience.” 


-Teacher 


“The pressure is on the administrators. They cannot transcribe everything that is going on in the 
classroom. Principals have to give a lot of evidence to give a 4. There is constantly a lot of 
confusion. There is a huge difference across schools as to what is a 4, and even within schools there 
is big variation in what is a 4.” 

-Teacher 


District leaders also share concerns about the inconsistency between schools within their districts. 
“There are different levels of quality with different administrators. When we look at the ratings 
from last year — ‘Highly Effective, Effective, and Ineffective’ — my weakest schools had the most 

“Highly Effective’ teachers.” 


-Superintendent 


“I think you will see consistency of the quality of SLOs within clusters but not across the district.” 


-Central Administrator 
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A key marker that affects consistency is whether the central administration and the school-based 
practitioners view implementation in similar terms. In some districts, the educators are very much on 
the same page regardless of their position. In other districts, there are markedly different opinions about 
TPE implementation between the central administration and the schools. 


For example, in one district where TPE is highly integrated into the instructional framework of the district, 
there are similarities of perceptions across different levels of the system. 


“They talk about data and instructional programs...So it forces that conversation. Teachers will tell 
you the same thing, that they have good conversations.” 


-Superintendent 


“How are SLOs affecting teacher/principal dialogue? From my previous experience as a principal, 
it was transformational. We can talk about data and get the bigger picture.” 


-Central Administrator 


“It’s a different dialogue than I had a few years ago. Before we had the SLO we would've talked 
about what the data looked like but no link to where the target is...The SLO has really shaped and 
forced those primary teachers to look at their data differently.” 


-Principal 


“There are more conversations with your colleagues. 
The principals spend more time having those 


meaningful dialogues on what you are doing in the 
classroom.” 


The more teachers are effectively 


supported, the greater their 
chances for success. 


-Teacher 


In contrast, comments from another district reveal opposing views on how TPE is being implemented. 
This district is experiencing greater inconsistency in the quality of implementation. 


“This evaluation system has helped us to build the capacity of people. It cannot be an event... You 
need to talk with the principals about consistency. There is a lot of professional development for 
principals and teachers.” 


-Superintendent 


“I would say there is more consistency than not, if a teacher moves from place to place they should 
have the same expectations — that is what we look for. When we see principals having a conference, 
are they using the rubric? If not, we push them to do so.” 


-Central Administrator 


“There is no consistency for our expectations for SLOs. I am not sure if there will be...I don’t 
believe there is any consistency across our county. We have very different views. We have 
principals who clearly state to teachers that ‘Highly Effective is only a place to visit and they 
don’t live there.’ I respectfully disagree with that. There is not a consistent message across the 
board.” 


-Principal 
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“The district did not provide a lot of training — most was given at the schools. It was all school- 
based from the administrators at the building. That was mostly about 2-3 years ago.” 


-Teacher 


Inter-rater reliability 


Many districts are increasingly using a quality rating rubric to provide a standard of quality for 
SLOs. They are using similar protocols to strengthen the observations of teacher practice. They are also 
engaging in formal training and team sessions to develop consistent approaches for rating SLOs and 
classroom observations. By working intentionally to improve the quality of inter-rater reliability, central 
and site level leaders find that their efforts are helping the districts to develop a common vocabulary 
about instruction. 


“How we deal with inter-rater reliability? We spent a great deal of time talking about that in our 
monthly principals’ meetings, at the director level meetings or at the larger curriculum meetings. 
We have crafted behavior stems to make sure people are on the same page. We have also worked to 
have a common lexicon of instruction, so teachers in one school hear the same thing as teachers in 
another school. We believe that if you don’t have a common language, you will likely be beat up by 


someone who likes wiggle room with vocabulary.” 
-Superintendent 


“We have invested heavily and given our principals a lot of professional development to show them 
the look-fors, to get inter-rater reliability as groups. We looked at video tapes of lessons, what did 
you see here, how can you match what you saw with the rubric, how to get greater consistency and 
better agreement on what is happening. That has all been very positive. One limitation is that you 
have such a wide variance in who is actually doing the classroom observation and what they were 


seeing. So that process has been helpful to us.” 
-Superintendent 


“We meet with instructional directors and pull a sample of SLOs and go through the rating 
process: why this one is okay, why this is stronger with this calibration, why this SLO is at a very 


low level.” 
-Central Administrator 


Yet there are also some districts that place less priority on developing consistency. 
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“We did training at the beginning of the year on inter-rater reliability but not since.” 


-Superintendent 


“We often have two different sets of eyes seeing dramatically different things in the same teacher 
and student group. It goes back to that they are not well trained. I think the focus is just not there. 


The district is making no effort to bring inter-rater reliability.” 


-Union Leader 
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Consistent training to address school needs 


For many districts in Maryland, the key to developing consistent processes and inter-rater reliability 
within and across schools begins with understanding and then addressing the capacity needs of 
principals and teachers. 


As noted in Chapter 3, principals are learning more about the challenges of supporting TPE 


implementation. They are identifying the support they need from their respective districts to be more 
effective. Figure V-3 highlights the focus and level of need from the perspective of Maryland’s principals. 


Figure V-3. Principal views on their capacity building needs, 2016 


100% 
22% 20% 
75% 
50% 
65% 69% 
25% 
0% 
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Standards. school improvements in 
improvement — my instructional 
To increase my effectiveness as a planning. supervision. 
principal, there needs to be more 
support at my school in... mAgree Undecided mDisagree 


Where TPE implementation is going well in schools, educators highlight a key contributing factor: 
the principals’ ability to synthesize the TPE process and clearly communicate expectations and 
practices to teachers. This means that principals need support to fully understand all aspects of the 
SLO and observation processes and how those two components can be leveraged to improve 
instruction in their schools. For many districts, building the capacity of principals is pivotal to developing 
consistent TPE processes and ratings within and across schools. Many but not all districts are focusing 
on building such leadership capacity in the schools. 


“In my mind, we should begin with our 
principals — that has to be the focus of staff 
development, making sure our principals have . aoe 
lie i. anid sails ae ne aia ae well in schools, educators highlight a 
leaders. I want all my principals to be key contributing factor: the principals’ 
instructional leaders. I want the central office ability to synthesize the TPE process 
to support the principals to get the job done and clearly communicate expectations 
and develop as instructional leaders.” and practices to teachers. 


Where TPE implementation is going 


-Superintendent 
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“Principals across schools are becoming more consistent. They are doing a better job across the 
board regardless of which school the teachers are in because there is a concerted effort to do that. I 
don’t want them to say, “we've done that, and now we are done.’ Everyone needs reminders and 


review and I hope they keep that up. I see a big push from the superintendent. I think he will 
continue in that direction.” 


-Union Leader 


When districts neglect the importance of building the capacity of principals, the result is weakened and 
inconsistent implementation. 


“The support for principals does not exist here...Why are we setting principals up to be the bad 
guys? Why are we setting schools up to fail? The district's messaging is not clear.” 
-Principal 


“The pressure is on teachers and principals. Where is the pressure on the central office to support 
me and my colleagues?” 


-Principal 


In many districts, teachers have the same capacity building needs as principals. They are also identifying 
the support they need, as Figure V-4 illustrates. 


Figure V-4. Teacher views on their capacity building needs, 2016 
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An inconsistency of implementation is that while many districts are leveraging the resources and 
methodologies provided by the state to increase teacher capacity, others are not. As a result, the 
training sponsored at the state level is reaching some districts and schools more than others. Whether 
this happens depends on the priorities of district leadership. 


“We looked at results around the state and how we fit in. We reviewed the highs and lows in each 
year. We met with our administrators and did some graphing of the way the SLOs were 
implemented and ranked according to our building. We were also supported to further those 
conversations in the building to make sure needs were being met in areas of assistance, 
inconsistencies, etc.” 

-Teacher 


“Teachers and principals should be trained together because there is always a gap — something is in 
or out, or part of it or not. If you put them together, for the majority, they should learn together. It 
will take away so many debates as the process is implemented through the year. There is no 
misunderstanding.” 

-Union Leader 


“I know what the district should have brought back to us, but the problem is the district didn’t 
bring back what they got from the training from MSDE.” 
-Teacher 


“No one helps you unless you reach out for help. You have to be self-dependent. There is a feeling of 
hopelessness in the first years and that is indescribable.” 
-Teacher 


Summary 


Achieving consistency of implementation at a high level of quality is an ongoing challenge within and 
across districts and schools. Districts that are building a consistent, quality implementation across 
schools are doing so through strategic efforts. For example, their leaders make a concerted effort to 
build both principal and teacher capacity. They build a consistency of understanding among educators 
at central and school site levels so that the district has a shared view of TPE implementation. They use 
rubrics to support the quality of SLOs, and the quality of observations. Simply put, they emphasize the 
importance of ensuring buy-in, developing consistency, and improving the quality of implementation. 
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Chapter Six: Manageability 


District leaders need to make decisions in a number of critical areas to effectively manage the TPE 
process. How districts balance the tradeoffs between a manageable system and a rigorous process 
requires careful consideration of what students and teachers need to fulfill district instructional goals. 
Districts have to decide how much flexibility to allow for school-to-school and classroom-to-classroom 
differences in student needs, while maintaining comparability and trust in the TPE system. The leaders 
who are effective at implementing TPE make it a district-wide priority to thoughtfully support and 
manage the evaluation system. 


Even in the growing number of districts with more advanced TPE practices, the leaders acknowledge 
there are still areas where they need to get better. By contrast, the districts with weaker implementation 
are giving these areas less meaningful attention. The areas are described below. 


Setting clear expectations for TPE to improve instruction 


Many districts have found that the best route to effective implementation is to make instructional 
improvement the primary focus of the TPE system. In these districts, management provides clear 
expectations for principals and teachers and uses TPE to improve instruction. 


Figure VI-1 shows marked differences between districts in terms of how teachers perceive the clarity of 
expectations for TPE. It presents the results for each of four quartiles of districts on the percentage of 
teachers who agree that the expectations are clear. The difference of 17 percentage points between the 


top quartile (where 78% of teachers agree that the expectations are clear) and the lowest quartile of 
districts (where 61% of teachers agree) is substantial. 


Figure VI-1. Teacher views on expectations of TPE, grouped by district quartiles, 2016 


Percentage of teachers agreeing that "the expectations are clear..." 


4th Quartile of Districts 


0% 10% 20% 30% 40% 50% 60% 70% 80% 
Note: The above graph shows districts grouped in quartiles by their level of agreement on select survey item(s) related to TPE 


expectations. Districts grouped in quartiles in this graph may not be the same as districts grouped in quartiles in the other graphs 
in this report. 
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When the instructional expectations for TPE are clear, it is readily apparent at district and school levels, 
and affects how implementation is managed. 


“Now all students count. The focus is on the 
progress the students made, what changes 
are needed, and how to make students 
is to make instructional improvement the succeed, It’s a huge change, happening in 

primary focus of the TPE system. all content areas.” 


The best route to effective implementation 


-Superintendent 


“Our principal is very organized and works diligently and conscientiously to provide clear 
expectations and support for quality instruction.” 
-Teacher 


“I think that's the value of the evaluation system: TPE helps us to manage the pace of 
improvement.” 
-Superintendent 


When a district is not clear about the purpose of the TPE system, teachers do not see the link between 
TPE and instructional improvement. 


“Teachers observe that that time hasn’t resulted in anything, not new knowledge, not agreement to 
practicing something different tomorrow...I am afraid that sometimes it might be more about 
compliance, and less about effectively changing the practice.” 


-Superintendent 


“I don't see TPE helping us instructionally. Because it has not been well defined on what the 
expectations are and what to accomplish instructionally.” 
-Union Leader 


Absent the instructional link, TPE becomes a managerial burden. It is perceived as a compliance 
activity rather than one that changes practice. What then gets implemented is a system that is either 
misunderstood, or misused to support only the accountability goals of a district. Most importantly, the 
opportunity for instructional improvement is lessened significantly. As one principal notes, “We are 
missing the spirit of what this evaluation system is trying to accomplish...and missing the conversation.” 


Making district support the priority 


A critical responsibility of districts is to create systems that support consistent, high-quality evaluations. 
As shown in previous chapters, that includes training principals and teachers on the district vision for 
instruction, on the components of the evaluation system, and on how the evaluation system supports 
the instructional goals of the district. When districts are able to manage TPE as an instructional 
reform, TPE provides the glue across instructional initiatives. In districts that build capacity in the 
schools and trust with their associations, professional development is focused and often a shared 
activity. 
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“We start at ground zero. We have to build their capacity. They come in without knowing anything 
about the process. Training takes place at the school level. The instructional supervisors have 
provided the training at schools. It helps new teachers to become instructionally strong.” 


-Superintendent 


“The conversation we are having with the principal today is: ‘Who else is doing the work? How are 
you building their capacity?’ We calibrate our feedback. We definitely agree that as a district we 
are taking a leadership approach to building capacity, not just the principals or the assistant 
principals.” 

-Central Administrator 


“As a union, we do several trainings per year and every time we get up to 100 teachers to go 
through the training. We used the same training the board uses. We used the same people, but I 
think it’s got to be targeted in the buildings. The district really supports professional development 
in terms of dedicating those days and time. I know everyone is fighting for time for more PD. 
From my perspective, the TPE professional development drives the ship.” 


-Union Leader 


The substance of meaningful communication, not just its appearance, is also part of a well- 
managed TPE process and can open the door to other levels of support. 


“Open communication, team-based planning, regular staff meetings, rotating stations of school- 
wide professional development, and observations have all been tremendously beneficial in our 
school's implementation of TPE.” 

-Teacher 


In each of the annual studies of TPE implementation in Maryland,'? educators indicate that they become 
more comfortable with TPE with practice, and time demands are eased as districts develop better 
managerial processes and support systems. 


Yet in an effort to make TPE more manageable or to ease workloads, a few districts are eliminating key 
steps that are essential to effective implementation. As an example, some districts reduce the 
opportunity for teachers and principals to engage in meaningful dialogue about instruction and student 
needs. 


“I never had conversations with my principal or my assistant principal about instruction, and 
never received a post-evaluation conversation. When I asked my principal when we can have a 
post-conference, he said that he only talks with people who “need the conversation.’” 


-Teacher 


“We are never allowed to have authentic conversations.” 
-Teacher 


“What I worry about is the cookie cutter process...My concern is that those conversations and 
dialogues with teachers should really be focusing on how we change this, and how we improve 
that. When we start the clicking, the process becomes an assembly line.” 


-Principal 
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When streamlining the implementation process, some districts are, in effect, undermining the 
potential impact of TPE on quality practice. This is a systemic problem which can also include 
eliminating steps that research has shown’ as integral to better instruction. For example, central 
administrators in a few districts determine what teacher SLOs will be, regardless of the needs of 
students in the respective classrooms. Some districts undercut other important steps in SLO 
implementation, such as eliminating the process of teachers identifying learning content and standards 
based on student need. These practices reduce teacher buy-in, lead teachers to conclude that SLOs are 
a waste of time, and make teachers feel that they are not being respected as professionals. The net 
effect is that the focus of SLOs becomes a compliance activity rather than one that drives instructional 
improvement. 


“SLOs in my county are mandated through and by central office. The assessments used are 
mandated, the amount of growth is mandated, and the information returned is exclusively 
controlled by central office. It does not shape instruction.” 

-Teacher 


“Our system does not let us select appropriate learning content for our SLOs as they are already 


predetermined at the county level.” 
-Teacher 


“Our county has had no support with developing SLOs. When we submit an SLO, we are told it is 
incorrect and the administration writes one for us. They set the goals to attain at 100% no matter 


what the baseline data reflects.” 
-Teacher 


Paying attention to the principal evaluation 


Principals have full plates in Maryland. One mechanism that districts have for understanding and 
addressing the needs of principals is to approach the new principal evaluation system as a 
serious endeavor. That’s what the stronger districts are doing, while others are focusing 
disproportionately on teacher evaluation and giving short shrift to the principal evaluation. In the districts 
more focused on the needs of principals, administrators are able to talk about the power of the principal 
evaluation. 


“When done correctly, you get to know the person — the strengths and weaknesses, engage that 
person prior to the evaluation. Otherwise, it’s cold. To evaluate principals, you need to visit the 
schools. We have teams from the central office. We have a pre-conference, look at their data, talk 
with the kids. Sometimes half a day, and sometimes a whole day. Sometimes we do follow-up visits 


based on the needs or conditions.” 
-Superintendent 


“The supervisors are having really meaningful conversations with principals about the quality of 
the TPE process and giving principals feedback. Good principals give feedback and get feedback, 
and their feedback loops here. There is an instructional coaching process.” 


-Central Administrator 
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As discussed in previous chapters, the quality and manageability of TPE implementation depends on the 
ability of principals to synthesize initiatives and lead the charge. With the new TPE system, the adoption 
of Maryland’s College and Career-Ready Standards, and the introduction of a new statewide 
assessment, the work and role of the principal are changing dramatically. 


It is the principal who is largely responsible for creating a coherent process in the school that leads to 
improved instruction. As one principal noted, “Nothing is being taken off the plate, and the plate is 
getting bigger.” When approached seriously, districts use the principal evaluation system as a way to 
better engage and support principals in their instructional leadership role. Conversely, without sufficient 
support, principals can feel ill-prepared to guide TPE implementation. 


“Somebody has to be the driver. I am depending on the principals to keep this process in place.” 


-Superintendent 


“We did have quite a bit of training at the beginning. What bothered me most is once I got the 
training, I became the expert in my building. I feel inadequate to be the expert. How to create 
SLOs of high quality or measurable, it’s new to me. I appreciate the professional development 
opportunities and I had an understanding. But I certainly should not be the one delivering that 
to my staff.” 


-Principal 


Understanding the value of technology and information management 


Technology platforms affect the quality and manageability of TPE implementation. When the technology 
works well, it helps facilitate conversation about instruction and enables districts to identify recurring 
challenges across multiple classrooms or schools. When it works poorly, people dismiss TPE as “too 
time-consuming” or “tedious.” Educators tend to blame reforms themselves for the faultiness of their 
implementation. Accordingly, if the technology is contributing to that faultiness, then it affects the 
perceptions as well as the manageability of TPE. 


The value of the technology should be to help districts generate information from SLOs, observations, 
and teacher and principal effectiveness ratings to inform their decisions about professional and 
leadership development, human capital, curricula, and assessments. Such technological capability 
should make it possible for districts to use the information from TPE to manage implementation more 
strategically. 


Some interviewees share examples of how TPE data systems are being used. 


“We use [name of data system]. We are really brought in and it drives the timing of the process and 
the conversations. I think it formalizes a lot of that process so we don't get slippage.” 
-Superintendent 
“For professional practice, you can pull that out, look at different domains and do trend analysis. 
For SLOs, it becomes really a repository. You can see what is done and not done.” 


-Superintendent 
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Many other interviewees, though, indicate that having a technology system does not mean that they can 
produce the type of information that would help inform decisions at the district or school level. Districts 
are still largely using the systems as repositories of information for the evaluation process, not as 
management tools. 


“Iam not all that knowledgeable about our system that we use today, but what I do know is that it 
is not a tool we use on a regular basis. It is more a depository, a place we keep forms and notes.” 


-Superintendent 


“Do I have a way to capture data to plan my summer professional development? Good question. 
My initial response is Iam not aware of it. I could be wrong. I just looked at the preliminary plan 
this morning. Iam not aware that anyone has ever said to me that according to teacher evaluation 
our people need help with X. That will be an ‘ah ha’ moment for me. I don't think we have any way 
to do that.” 

-Superintendent 


One of the biggest frustrations among educators is when the technology does not support an efficient 
TPE process, yet adds to the workload. 


“The observation process is also getting 
so bogged down that people get Educators tend to blame reforms themselves 
frustrated to use it. They get kicked out for the faultiness of their implementation. 

of the system, or it does not print out Accordingly, if the technology is contributing 
what they need, or printed extra to that faultiness, then it affects the perceptions 


pildaaad Melia SUES ny as well as the manageability of TPE. 
feel is a smooth process. : 


-Superintendent 


“There is a gap when teachers have no access to SLOs. We had to submit online, but then we could 
not access the online system for a while. We were then blind for our goals for a period of time. My 
principal just emailed me a PDF copy of my goals because I requested it. I couldn't access them 
online. So if you forgot any aspect of it, you could not look it up.” 


-Teacher 
“It is like we are doing double work. We kept collecting data and we have to put it in the other 
system. We are doing the same thing over and over again. It creates more work.” 


-Teacher 


Summary 


To manage TPE well, districts need to do several things well: provide clear expectations that TPE is 
focused on improved instruction; train teachers and principals on high quality implementation strategies; 
create a meaningful principal evaluation system to better understand and address principal needs; and 
develop technology tools to simplify the process and generate information to inform decisions about 
student, teacher, and principal needs. 
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Chapter Seven: Transitions 


Change is constant and far-reaching for Maryland’s schools and districts. Recent initiatives include 
implementing TPE, introducing College and Career-Ready Standards, developing curricula based on 
these new standards, and launching the new Partnership for Assessment of Readiness for College and 
Career (PARCC) assessments. All of these initiatives involve changes of practice and, when well 
implemented, changes of culture. 


Throughout the introduction and early implementation of these changes, the state’s core strategies 
emphasize improved instruction, capacity building, collaboration among diverse stakeholders, 
and better results. Many districts and schools have taken approaches which build on these same 
strategies. They have also worked with school level educators to get past fears and initial perceptions 
regarding, in particular, the intent and use of the new evaluation system and new assessments. 


The level of change continues, affected by federal and state factors. For example, the federal Every 
Student Succeeds Act prohibits the U.S. Secretary of Education from mandating the elements of 
educator evaluation systems, while allowing the use of Title I-Part A funds for the development, 
refinement, and implementation of the evaluation systems.'* The next steps for TPE implementation 
under ESSA are now in the hands of the state and districts. 


Concurrent with the federal changes, there are also changes in state leadership at the policy level with 
many new appointments to the Maryland State Board of Education, and at the executive level with the 
appointment of a new State Superintendent of Schools. 


These changes create uncertainties for educators in Maryland’s schools and districts. In a changing 
environment, educators are concerned about the future of TPE and the direction and processes that will 
be pursued moving forward. 


“The concern I have is that with the political The state’s core strategies emphasize 
climate changing and the changes in the state improved instruction, capacity 
board, I sincerely hope this evaluation system : 
doesn't get trashed and we are left floundering 
in a stormy sea.” 


building, collaboration among diverse 
stakeholders, and better results. 


-Superintendent 


“We are concerned that we are going down this path and then we'll be told the path will take a 
detour. We are now worried about everything changing. People don’t know what the focus is 
going to be or where the support will be.” 

-Superintendent 


“I am afraid to say to teachers that ‘now there may be more changes.’” 


-Union Leader 


“Not having stability at the state level is concerning. Our message to our folks is that our focus is 
great teaching for every student every day. We provide support to that. With TPE, we focus on 
what we know is good instruction that leads to good learning.” 


-Superintendent 
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In interviews, district and school leaders are characteristically making the same statement—they want 
the evaluation system to continue to focus on instructional improvement. Many districts have been 
working to change their practices and cultures. There are a number of other districts that are now 
moving in these directions. On a recurrent basis, many leaders and practitioners are underscoring 
two key points —stay the course in the focus and support of TPE, while allowing more flexibility in 
the weighting of the components. 


“I would like to see SLOs remain. That for me has the greatest chance of tying everything back to 
the whole focus, the whole idea of continuous improvement. Without that, it’s just focused on 
professional practice, not holding to the accountability piece.” 


-Superintendent 


“There is a lot of discussion about changing the system. But we just want to keep going so stay the 
course.” 
-Central Administrator 


“We will continue what we are doing because we see the value — not because it’s mandated. I would 
be really disappointed if SLOs are gone. I think if I told my teachers we don’t need to do SLOs, 
most of the teachers would still do them. They see the value of it.” 


-Principal 


“We will keep SLOs because that’s the one thing we've seen in teacher evaluation that offers 
opportunities for dialogue, reflection, and concrete suggestions for improvement, instead of a 
checklist of things.” 


-Union Leader 


The weighting of the components in TPE is a pivotal issue for Maryland districts. The majority of 
interviewees indicate that they would want to adjust the percentage of the evaluation that focused on 
student growth. In addition, others note the need for modifications in the evaluation cycle. 


“Tam not a fan of student growth at 50% of teacher evaluation. Data should definitely be a piece, 
but 50% has created some issues for us. If they keep it at 50%, my people will not be happy. We 
don't know what the state is going to do. We don’t know if the state board will turn and go and 
take a different direction.” 

-Superintendent 


“We used the state model, 50%-50%. We have come up with a 70-30 split — we are looking at 
professional practice for 70%, and 30% for SLOs.” 
-Union Leader 


“The weighting of SLOs irritates the hell out of me. I agree there is a place for SLOs, but the weight 
is not right.” 
-Principal 


“I tend to think the SLO is weighted too heavily but I would not change the structure. It has 
enhanced our practice. The goal setting is more intentionally focused.” 


-Central Administrator 
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The process for how districts might approach making mid-course modifications is also the basis for 
comment. 


“What will I change if I can do anything with 


TPE? I will go back to my task force. We will Many leaders and practitioners are 
re-look at the observation process, and we will underscoring two key points — stay the 
look at how we use SLOs in concert with the course in the focus and support of TPE, 
P ron aullined RACHCE baa a while allowing more flexibility in the 
adjustment is needed. ce. i : 
weighting of the components. 
-Superintendent 


“The way forward is to make minor changes to 
allow the local systems to do what is best for their students. My feet are just getting wet on this. 


I think TPE is valuable.” 
-Union Leader 


“I hope we are pushing this forward based on what we have in the evaluation system that is leading 
to better conversations between educators and administrators.” 


-Union Leader 


“I think we will keep the evaluation process exactly as it is.” 
-Superintendent 


Some interviews show an undercurrent of opposition to the use of student academic growth in 
TPE. While these positions can sometimes be confrontational, they can also provide the basis for 
discussion and negotiation related to student growth and the importance of having multiple measures to 


gauge student progress. 


“Without a doubt, we should decouple student performance from teacher evaluation.” 


-Union Leader 


“On TPE, there is already pushback from the teachers’ association about not using student 
outcomes as part of teacher evaluation. Well, on the one hand, I understand their position, but I do 
believe teachers have the ability to make an impact on how students do on tests. I think people I 
have worked with have shown that intentional work on student outcomes can improve those 


student outcomes.” 
-Superintendent 


“We should be able to use multiple sources of achievement data to demonstrate our students’ 


growth.” 
-Teacher 


“Are we wedded to 50% on outcome? Do we want to give up something we believe that might 
enhance our outcomes? I think there are places we can do better, but I am not too interested in 
negotiating this with the association. I would rather have a conversation with the union to see if 
we can come to a reasonable place where student growth is still part of it.” 


-Superintendent 
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Summary 


The state’s core strategies to support TPE implementation—emphasizing improved instruction, capacity 
building, collaboration among diverse stakeholders, and better results—are being replicated by many 
districts and schools. Recent changes in federal policy and state level leadership are creating 
uncertainties for educators in Maryland. In a changing environment, educators are concerned about the 
future of TPE and the direction and processes that will be pursued moving forward. On a recurrent basis, 
many leaders and practitioners are underscoring two key points—stay the course in the focus and 
support of TPE, while allowing more flexibility in the weighting of the components. 
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Chapter Eight: Issues and Recommendations 


Maryland has made significant progress in the early years of implementing the new teacher and principal 
evaluation systems. The key to continuing progress is for district leadership to focus on managing 
constructive change, by developing principal and teacher leadership and emphasizing instructional 
improvement. 


The charge for leaders is to develop leadership in others and to put teachers in a position to be 
more effective with more students. Increasingly in Maryland, there are districts whose central and 
school site leaders are managing these kinds of changes and systems, and doing it well. These leaders 
are supporting and advancing instructionally-focused evaluation reform. There are a smaller number of 
districts whose leaders, whether through insufficient attention or ineffective implementation processes, 
are failing to place priority on thoughtfully strengthening 
practitioners and on effecting instructional linkages to 
evaluation. In essence, they are lessening the potential 
impact in their districts of the statewide efforts to improve The charge for leaders is to develop 
educator performance and evaluation. leadership in others and to put 


teachers in a position to be more 
effective with more students. 


The recommendations focus on strengthening leadership 
and further improving implementation at central and 
school site levels within Maryland’s districts. 


Issue One: Statewide implementation of change 


Overview 

There are major state level implications of the TPE-related work conducted to-date. The state should use 
learnings to inform next steps under the Every Student Succeeds Act. Further, the state should use TPE 
as a model for statewide implementation of other major reform initiatives. 


Recommended action 


Base the ESSA evaluation strategy on TPE’s progress. With each succeeding year of implementation, 
Maryland’s educators are more supportive of and confident in TPE. Many districts are growing in their 
ability to support implementation. The state’s building block approach to developing capacity and 
increasing the constituencies in support of TPE is proving productive. ESSA provides the state with the 
opportunity to continue TPE’s dual emphases on professional practice and student growth, while 
revisiting the weighting of the components. 


Renew the eight organization alliance in support of evaluation and instructional improvement. By 
coalescing all of the major educational entities in the state, Maryland is showing the country a way to 
advance evaluation reform through focused collaboration. The key constituencies in the state have been 
working together on behalf of students, teachers, and principals. With the initial Memorandum of 
Understanding having concluded in June 2016, MSDE should again assume a leadership role in ensuring 
that this statewide collaboration continues. 


Use the learnings from TPE implementation to inform new statewide initiatives. MSDE’s strategy for 
implementing TPE is multi-faceted: engage key stakeholders as partners, provide professional 
development regionally and statewide, promote open and two-way communication, prepare future 
practitioners, and make mid-course improvements based on changing needs in the field. These 
practices are contributing significantly to progress in TPE implementation. These same practices can be 
pivotal to the success of future initiatives in the state. 
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Impact 

Maryland is a leader in implementing new educator evaluation systems. Maryland’s success to-date 
is based on substance and process. It should shape future TPE implementation under ESSA and inform 
other statewide reform initiatives. 


Issue Two: District support capacity 


Overview 

Increasing the quality, consistency, and manageability of TPE requires thoughtful and aligned actions at 
central office and school site levels. Advancing higher levels of practice, by improving the quality and 
coherence of support to the schools and classrooms, does not happen accidentally. The Maryland 
districts with better systems of support are making concerted effort to improve and refine their 
implementation of the new evaluation systems. 


The role of district leadership is paramount in the TPE process. Where progress is being made, it is a 
direct function of the organized capacity of the district to support the school sites, integrate instructional 
initiatives, and focus the central administration on better practices. 


There is greater consistency and quality of TPE implementation when the district emphasis is on 
engagement and collaboration, professional development and leadership development, and instructional 
improvement. In the districts that lack such emphasis, there is less buy-in and fidelity of implementation. 


Recommended action 

Provide advanced training to school site and district level practitioners. Teachers, principals, and central 
administrators need increased opportunities to be the beneficiaries of field-proven tools and processes 
to support implementation. They are now identifying needs in areas such as the use of quality rating 
rubrics, ways to advance inter-rater reliability, methods for integrating SLOs, observations, and 
Maryland’s College and Career-Ready Standards, and management strategies for achieving greater 
quality and consistency within and across schools. 


Districts should replicate the MSDE model of providing training to cross-level teams. This approach 
enables educators at different levels of the system to hear and learn from the same things in the same 
ways—thus contributing to a common language and shared understanding of best practices for 
strengthening the implementation of TPE. 


Focus on instructional improvement to increase accountability. Emphasizing the direct link of TPE to 
instructional improvement is pivotal to success. It encourages the thinking process for teachers and 
principals on issues of substance: learning content, instructional strategies, and student growth. When 
this direct link is lacking, a district loses the power of the reform. The result is compliance-driven activity, 
with little in the way of true support or accountability. 


Emphasize teacher voice and constituency building. The more effective Maryland districts understand 
that meaningful implementation of TPE requires building the capacity of key constituencies to 
understand and carry out the reform. This can only happen when district leaders take TPE seriously, 
articulate the connections between TPE and other major instructional initiatives, and listen to the voices 
of the schools. Implementation continues to flounder in districts whose leadership either takes a top- 
down or a benign neglect approach to the schools. 


Even when the process can be difficult, district leaders need to keep talking with teachers. Doing so 


requires having vehicles for two-way communication and being anticipatory in understanding the 
importance of teacher trust when undertaking new initiatives. Four years of survey and interview data 
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show that the districts which have strong, two-way 
communication with teachers and teacher leaders, even if : ous 
it does always take the form of formal collaboration, are Nurturing a new direction for 
achieving higher levels of teacher buy-in. Where it is instructional improvement and 
lacking, the districts are struggling and, characteristically, evaluation requires leaders who 
have more the appearance than the substance of real 
communication. 


can change practices, mindsets, 
and the culture of central offices 


Engage the local boards of education. Implementing new and schools. 
evaluation systems is a policy decision that triggers other 
policy decisions. It is not enough for local board members 
to know that a new evaluation system is being implemented. They need to know what findings and 
organizational learnings are emerging from the new evaluative data that have implications for district 
policy in such areas as professional development, assessments, and budgetary priorities. 


Conduct a thoughttul reset in select districts. Based on learnings from MSDE- and MSEA-sponsored 
training sessions, and bolstered by customized on-site support, several districts that initially got off to a 
slow start have made major strides forward in implementing TPE. Survey and interview data underscore 
that there are some districts that need to follow a similar course of action and learn from the better 
practices being used in other Maryland districts and nationally. 


Impact 


Nurturing a new direction for instructional improvement and evaluation requires leaders who can 
change practices, mindsets, and the culture of central offices and schools. Maryland’s progress to- 
date provides the foundation for the steps which need to be taken to deepen the impact and reach of 
TPE. 


Issue Three: Principal leadership 


Overview 


Principal leadership is key to the success of any serious initiative at a school site. Implementing the new 
teacher evaluation system is placing great demands on the time and skills of principals. Principals need 
focused support to do this work well. In this same regard, they also need a principal evaluation system 
that can strengthen their performance and contribute to their growth as school leaders. 


Recommended action 

Increase the capacity of principals to guide and oversee implementation. Districts need to provide 
principals with more coaching and formalized support as they implement the new evaluation systems. 
Meeting principals’ needs has to be approached more systematically than it is in many districts. 
Specifically, principals need more coaching in how to lead instructionally-focused conversations with 
teachers, how to organize schools for greater quality and consistency in TPE implementation, and how 
to support teachers in setting and learning from student growth targets. 


Part of this capacity building is to recognize that many principals are feeling overwhelmed. If they are to 
give meaningful attention to their responsibilities in leading instructional and evaluation reform under 
TPE, their workloads need to be adjusted in other areas to reflect the demands of their new 
responsibilities. 
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Make principal evaluation a greater priority in districts. Districts need to give attention to the quality of 
and priority for their new evaluation systems for principals. While receiving ratings is part of evaluation, it 
is not the system itself. An effective evaluation system must contribute to the professional growth of 
stronger principals, while providing support to improve the performance of other principals. Districts 
need to devote time and concerted effort to building in more focus and support to their principal 
evaluation systems. 


Impact 

The credibility and usefulness of a new principal evaluation system lies in whether it recognizes 
and provides the vehicle for improving performance. Principals of all performance levels need an 
evaluation system that produces these outcomes. Moreover, teachers have to see that principals are 
getting better as school leaders. 


Summary 


Maryland is making significant progress in implementing the new teacher and principal evaluation 
systems, as reflected in survey and interview data over a four-year period. Moving forward, district- 
and school-based educators indicate that Maryland should stay the course in the focus and 
support of TPE, while allowing more flexibility in the weighting of the components. This means 
continuing to link TPE to instructional improvement and continuing to strengthen the quality, 
consistency, and manageability of implementation. 
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Appendix 


Figure 1: State teacher and principal evaluation models 
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at the classroom level 
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Math or 20% Reading 

15% Annual SLO Measure 
as determined by 
priority identification 
at the district or 
school level 

15% Annual SLO Measure 
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at the classroom level 


20% SLO Lag Measure based on 
School Progress Index 
Indicators ( Achievement, Gap 
Reduction, Growth, College and 
Career Readiness), Advanced 
Placement Tests, or similarly 
available measures 
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the district or school level 
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Reading and 10% 
Math 
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MSA and 10% Math MSA 

10% School Progress Index 

10% Annual SLO Measure as 
determined by priority 
identification at the district 
level 

10% Annual SLO Measure 
as determined by 
priority identification at 
the school level 


High School 
Principals 


20% SLO Lag Measure as determined by 
10% HSAs and 10% AP scores, 
SPI Indicators (Gap Reduction, 
College & Career Readiness, 
Achievement), or similar valid 
delayed measures 

10% School Progress Index 

10% Annual SLO Measure as determined 
by priority identification at the 
district level 

10% Annual SLO Measure as determined 
by priority identification at the 
school level 


Other Principals (e.g., Special 
Center, PreK-2) 


20% SLO Lag Measure as determined by 
10% HSAs and 10% AP scores, 
SPI Indicators (Gap Reduction, 
College & Career Readiness, 
Achievement), or similar valid 
delayed measures 

10% School Progress Index 

10% Annual SLO Measure as determined 
by priority identification at the 
district level 

10% Annual SLO Measure as determined 
by priority identification at the 
school level 


Source: 7he Maryland Teacher and Principal Evaluation Guidebook, Version 3, 
http://archives.marylandpublicschools.org/tpe/TPE_Guidance_Version3_092013.pdf 
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Figure 2: Local teacher and principal evaluation models 
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Source: 7he Maryland Teacher and Principal Evaluation Guidebook, Version 3, 
http://archives.marylandpublicschools.org/tpe/TPE_Guidance_Version3_092013.pdf 
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Table 1. Survey response rates by district, 2016 


District Response Rate 
Allegany County 45.5% 
Anne Arundel County 47.9% 
Baltimore City 10.2% 
Baltimore County 13.5% 
Calvert County 32.7% 
Caroline County 37.0% 
Carroll County 14.1% 
Cecil County 69.1% 
Charles County 62.5% 
Dorchester County 48.6% 
Frederick County 23.8% 
Garrett County 51.8% 
Harford County 34.6% 
Howard County 26.6% 
Kent County 43.2% 
Montgomery County 31.6% 
Prince George's County 58.1% 
Queen Anne's County 85.3% 
Somerset County 63.4% 
St. Mary's County 30.0% 
Talbot County 38.9% 
Washington County 36.5% 
Wicomico County 33.5% 
Worcester County 34.3% 
State Total 34.6% 


Note: Statewide, the number of survey respondents increased by 16.6% (or 2,708 educators) from 2014 (i.e., 16,314) to 2015 (i.e., 
19,022), and by 15.2% (or 2,894 educators) from 2015 to 2016 (i.e., 21,916). 
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Table 2. Characteristics of the survey respondents, 2014-2016 


| have*: 
Participated in classroom observation training(s) 


Participated in Student Learning Objectives (SLO) training 


Accessed other source(s) of information or experiences related to TPE 


Accessed other source(s) of information or experiences related to SLOs 


The school level | work in is: 
Elementary school 
Middle school 
High school 
Not Applicable / No Response 
My position is described by this educator category: 
Classroom teacher 
Non-classroom teaching staff 
Special educator 
Principal 
Assistant principal 
Other administrator 
Central office staff 


No Response 


2014 
(n=16,314) 


44% 
86% 
26% 
49% 


48% 
23% 
27% 

2% 


68% 
11% 
12% 
4% 
2% 
1% 
1% 
1% 


In my position, the majority of my work is in the following subjects and/or area(s)*: 


Arts 

Career and Technical Education 
Elementary (multiple subject areas) 
English/Language Arts 

English as a Second Language 
Health and Physical Education 
Math 

Science 

Social Studies 

Special Education 

World Languages 


Other (e.g., music) 
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8% 
6% 
31% 
21% 
3% 
6% 
18% 
15% 
14% 
13% 
4% 
8% 


2015 
(n=19,022) 


49% 
85% 
26% 
53% 


48% 

21% 

27% 
3% 


66% 
13% 
13% 
3% 
3% 
1% 
1% 
1% 


7% 
5% 
28% 
20% 
3% 
5% 
17% 
12% 
12% 
12% 
4% 
9% 


2016 


(n=21,916) 


50% 
83% 
27% 
54% 


49% 
23% 
26% 

2% 


68% 
12% 
12% 
3% 
2% 
1% 
1% 
1% 


7% 
4% 
28% 
20% 
4% 
6% 
17% 
13% 
12% 
12% 
4% 
8% 
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2014 
(n=16,314) 


2015 
(n=19,022) 


Years of teaching experience (teachers only; n=14,905 for Y2, 17,344 for Y3, and 20,155 for Y4): 


0-3 
4-6 
7-9 
10+ 


Not Applicable / No Response 


11% 
10% 
15% 
63% 

1% 


11% 
10% 
12% 
65% 

1% 


Years of experience as a principal (principals only; n=1,029 for Y2, 1,160 for Y3, and 1,235 for Y4): 


0-3 
4-6 
7-9 
10+ 
Not Applicable / No Response 
The highest educational degree | have attained is: 
Bachelor's degree 
Master's degree 
Doctoral degree 


No Response 


27% 
17% 
18% 
25% 
13% 


21% 
76% 
2% 
1% 


29% 
17% 
15% 
26% 
12% 


20% 
76% 
3% 
1% 


2016 


(n=21,916) 


12% 
10% 
11% 
67% 

1% 


26% 
19% 
14% 
29% 
12% 


20% 
76% 
3% 
2% 


Note: Due to rounding, percentages may not always appear to add up to 100%. “Denotes multiple choice survey items; 


percentages do not add up to 100. 
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Table 3. Responses from principals and teachers, 2016 


Systemic Context 


The expectations are clear for teachers and principals 
in the TPE system.” 


My district’s support of the implementation of the TPE 
system is helpful.* 
My school’s support of the implementation of the TPE 
system is helpful.* 


Quality of TPE Frameworks and Processes 
Teacher evaluation frameworks and processes... 


Provide useful feedback to teachers.* 


Encourage reflection on instructional practices.* 


Lead to improved decisions related to instructional 
approaches.* 


Classroom Observations 

Classroom observations... 
Are conducted by observers/evaluators qualified to do 
the evaluation.* 

Student Learning Objectives (SLOs) 

Teachers receive... 


Information about developing high quality SLOs.* 


Guidance in selecting learning content for SLOs.* 


Guidance in selecting research-based instructional 
strategies for SLOs.* 


Information on connecting Maryland's College and 
Career-Ready Standards to SLOs.* 
Recommendations for selecting pre/post assessments 
for SLOs.* 
Valid and reliable approaches to setting growth targets 
for SLOs.* 
Professional development that is informed by the SLO 
process.* 

This Year’s Experience 

This year, | believe... 
There were opportunities to confer with my principal 
about my SLOs.* 


There were opportunities to review and refine my 
SLOs.* 


Reflection on my instructional practices deepened.* 


There was more instructionally focused dialogue with 
my colleagues.” 


There was more instructionally focused dialogue with 
my supervisor.* 


The results of my evaluation will be used to inform my 
professional development plan for next year.* 


Agree 


88% 


88% 


91% 


89% 


89% 


83% 


98% 


85% 


87% 


76% 


71% 


19% 


77% 


73% 


90% 


91% 


77% 


79% 


81% 


76% 
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Principals 


(n=1,235) 


Undecided Disagree 


6% 


7% 


5% 


1% 


7% 


10% 


1% 


1% 


7% 


12% 


15% 


8% 


10% 


12% 


5% 


5% 


15% 


12% 


11% 


16% 


6% 


5% 


4% 


4% 


5% 


6% 


1% 


7% 


6% 


12% 


15% 


13% 


13% 


15% 


4% 


4% 


8% 


9% 


9% 


8% 


Agree 


68% 


57% 


63% 


62% 


12% 


58% 


81% 


64% 


63% 


52% 


51% 


57% 


54% 


53% 


68% 


15% 


54% 


54% 


48% 


51% 


Teachers 
(n=20,155) 


Undecided Disagree 


16% 


25% 


22% 


15% 


12% 


19% 


9% 


13% 


13% 


17% 


19% 


15% 


17% 


17% 


10% 


9% 


18% 


16% 


18% 


24% 


16% 


18% 


16% 


24% 


17% 


23% 


11% 


23% 


25% 


32% 


30% 


28% 


30% 


30% 


22% 


15% 


28% 


31% 


34% 


25% 
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Principals Teachers 
(n=1,235) (n=20,155) 
Capacity Building 
To increase my effectiveness as a teacher or a principal, Agree Undecided Disagree Agree Undecided Disagree 
there needs to be more support at my school in... 


Analyzing student data for action.* 71% 7% 22% 49% 15% 35% 
Using assessments to measure growth.* 70% 8% 23% 45% 15% 39% 
Using Maryland's College and Career-Ready 65% 13% 20% 44% 22% 34% 


Standards.* 


ee to Maryland’s College and Career-Ready 64% 13% 20% 43% 22% 35% 


Receiving SLO feedback from school administrators N/A N/A N/A 46% 17% 37% 
(for teachers only). 
Receiving SLO feedback from district administrators 57% 16% 27% N/A N/A N/A 


(for principals only). 


Using data from SLOs and observations to strengthen 69% 10% 21% AT% 20% 33% 
school improvement planning. 


Using data from SLOs and observations to make 
improvements in my instructional practices (for N/A N/A N/A 49% 17% 34% 
teachers only). 
Using data from SLOs and observations to make 
improvements in my instructional supervision (for 69% 11% 20% N/A N/A N/A 
principals only). 

Note: Principals refers to principals and assistant principals; 7eachers refers to classroom teachers, non-classroom teaching staff, 


and special educators. Agree is a composite of strongly agree/agree and Disagree is a composite of strongly disagree/disagree. 
* indicates statistically significant differences at the 95% confidence level. 
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Table 4. Responses from principals and teachers to same questions, 2014-2016 


Systemic Context 


The expectations are clear for 
teachers and principals in the 
TPE system. 


My district’s support of the 
implementation of the TPE 
system is helpful. 


My school’s support of the 
implementation of the TPE 
system is helpful. 


Year 


2014 
2015 
2016 
2014 
2015 
2016 
2014 
2015 
2016 


10% 
82% 
88% 
17% 
81% 
88% 
88% 
88% 
91% 


Quality of TPE Frameworks and Processes 


Teacher evaluation frameworks and processes... 


Provide useful feedback to 
teachers. 


Encourage reflection on 
instructional practices. 


Lead to improved decisions 
related to instructional 
approaches. 


Classroom Observations 
Classroom observations... 


Are conducted by 
observers/evaluators qualified 
to do the evaluation. 


2014 
2015 
2016 
2014 
2015 
2016 
2014 
2015 
2016 


2014 
2015 
2016 


Student Learning Objectives (SLOs) 


Teachers receive... 


Information about developing 
high quality SLOs. 


Guidance in selecting learning 
content for SLOs. 


Guidance in selecting 
research-based instructional 
strategies for SLOs. 
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2014 
2015 
2016 
2014 
2015 
2016 
2014 
2015 
2016 


84% 
87% 
89% 
89% 
90% 
89% 
19% 
82% 
83% 


90% 
93% 
98% 


70% 
76% 
85% 
16% 
19% 
87% 
63% 
70% 
76% 


Principals 
U D 
13% 17% 
8% 9% 
6% 6% 
13% 10% 
13% 7% 
7% 5% 
9% 3% 
8% 4% 
5% 4% 
11% 5% 
9% 4% 
7% 4% 
7% | 4% 
6% 4% 
7% 5% 
15% 6% 
13% 5% 
10% 6% 
6% 4% 
5% 3% 
1% 1% 
13% 17% 
9% 14% 
7% 7% 
10% 15% 
9% 12% 
7% 6% 
16% 22% 
13% 16% 
12% 12% 


Diff 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 


Teachers 
U D 
20% | 30% 
17% 20% 
16% 16% 
31% | 29% 
31% 23% 
25% 18% 
23% | 21% 
24% 19% 
22% 16% 
24% | 27% 
20% 26% 
15% 24% 
17% 17% 
14% 16% 
12% 17% 
26% | 26% 
23% | 24% 
19% 23% 
19% 20% 
16% 18% 
9% 11% 
16% 34% 
14% 29% 
13% 23% 
16% 34% 
15% 29% 
13% 25% 
20% 43% 
18% 37% 
17% 32% 


Diff 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 


Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 
Y2 vs. 
Y2 vs. 
Y3 vs. 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 


Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
Y3: 
Y4: 
Y4: 
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Principals Teachers 


Year 
A U D Diff A U D Diff 

Information on connecting 2014 55% 19% 26% Y2vs.Y3:* | 33% 23% 43% | Y2 vs. Y3:* 
Maplendis © oles and 2015 65% 17% 19% Y2vs.Y4:* 45% 20% 35% Y2vs. Y4:* 
Career-Ready Standards to 
SLOs. 2016 71% 15% 15% Y3vs.Y4:* 51% 19% 30% Y3vs. Y4:* 

; 2014. 71% 12% 18% Y2vs. Y3:* | 48% 16% 36% Y2-vs. Y3:* 
Recommendations for 
selecting pre/post 2015 75% 11% 15% Y2vs.Y4:* 53% 15% 31% | Y2vs. Y4:* 
assessments for SLOs. a 


2016 79% 8% 13% Y3vs. Y4: 57% 15% 28% Y3vs. Y4:* 
; . 2014 60% 15% 25% Y2vs.Y3:* | 40% 20% 41%  Y2-vs. Y3:* 
Valid and reliable approaches 
to setting growth targets for 2015 65% 15% 20% Y2vs.Y4:* 47% 18% 35% Y2vs. Y4:* 
res 2016 77% 10% 13% Y3vs.Y4:* 54% 17% 30% Y3vs. Y4:* 
: 2014 66% 15% 20% Y2vs. Y3:* | 48% 20% 37%  Y2vs. Y3:* 
Professional development that 
is informed by the SLO 2015 71% 13% 16% Y2vs.Y4:* 49% 18% 33% Y2vs. Y4:* 
alee 2016 73% 12% 15% Y3vs.Y4:* 58% 17% 30% Y3vs. Y4:* 


This Year’s Experience 
This year, | believe... 


adiciistoenteunliests 2014 90% | 5% 5% Y2vs.Y3: 71% 9% 20% Y2vs. Y3:* 
confer with my principal about 2015 | 88% 7% 5% Y2vs.Y4:* 67% 10% | 23% Y2vs. Y4: 

De ee 2016 90% 5% 4% Y3vs.Y4:* 68% 10% 22% Y3vs. Y4:* 
2014. 82% | 7% 11% Y2vs.Y3: 638% 11% 26% Y2vs. Y3:* 
2015 79% 9% 12% Y2vs.Y4:* 64% 12% 24% Y2vs. Y4:* 
2016 91% 5% 4% Y8vs.Y4:* 75% 9% 15% Y3vs. Y4:* 
2014. 72% | 19% 9% Y2vs.Y3: 44% 21% 36% Y2vs. Y3:* 
2015 71% 18% 11% Y2vs.Y4:* 50% 19% 32% Y2vs. Y4:* 
2016 77% 15% 8% Y8vs.Y4:* 54% 18% 28% Y3vs. Y4:* 
There was more instructionally 2014 75% | 16% | 9% Y2vs.Y3: 42% 18% 40% Y2 vs. Y3:* 
focused dialogue with my 2015 75% 16% 9% Y2vs.Y4:* 48% 17% 35% Y2vs. Y4:* 
RONSAaHES: 2016 79% 12% 9% Y3vs.Y4:* 54% 16% 31% Y3Vvs. Y4:* 
There was more instructionally 2014 78% 13% | 9% Y2vs. YS: 39% 19% | 42% Y2 vs. Y3:* 
focused dialogue with my 2015 | 78% 138% 9% Y2vs.Y4:* 44% 18% 39% Y2vs. Y4:* 
eupetieat: 2016 81% 11% 9% Y3vs.Y4:* 48% 18% 34% Y3Vvs. Y4:* 
The results of my evaluation 2014. 74% 20% 6% Y2vs.Y3: 45% 30% 25% Y2vs. Y3:* 
WUT De Head to anton h ny 2015 75% 17% 8% Y2vs.Y4:* 49% 26% 26% Y2vs. Y4:* 


professional development plan 
for next year. 2016 76% 16% 8% Y3vs.Y4:* 51% 24% 25% Y3vs. Y4:* 


There were opportunities to 
review and refine my SLOs.** 


Reflection on my instructional 
practices deepened. 


Note: Principals refers to principals and assistant principals; 7eachers refers to classroom teachers, non-classroom teaching staff, 
and special educators. A = Agree, a composite of strongly agree/agree. U/= Undecided. D= Disagree, a composite of strongly 
disagree/disagree. Y2 refers to 2014, Y3to 2015, and Y4to 2016. * indicates statistically significant differences at the 95% 
confidence level. N (principal, 2014 = 1,029; 2015 = 1,160; 2016 = 1,235); N (teacher, 2014 = 14,905; 2015 = 17,344; 2016 = 20,155). 
“In 2014 and 2015, this survey item read: “There were opportunities to engage in a mid-interval review and refinement of my SLOs.” 
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Table 5. Responses from principals and teachers to same questions, 2013-2016 
Principals Teachers 
A U D Diff A U D Diff 


Year 


Systemic Context 


2013 55% | 26% | 20% Yivs.Y4:* 41% 31% | 28% | Y1 vs. Y4:* 
The expectations are clearfor 2014 70% 13% 17% |Y2vs.Y4:* | 50% | 20% 30% Y2vs. Y4:* 
teachers and principals in the 
TPE system. 2015 82% 8% 9% Y3vs.Y4:* 638% 17% 20% Y3 vs. Y4:* 
2016 88% 6% 6% 68% 16% 16% 


Quality of TPE Frameworks and Processes 
Teacher evaluation frameworks and processes... 


2013 | 66% | 24% | 10% Y1vs.Y4:* 38% 38% 25% |Y1 vs. Y4:* 
Lead to improved decisions 2014. 79% 15% | 6% Y2vs.Y4:* | 48% 26% 26% Y2vs. Y4:* 
related to instructional 
approaches. 2015 82% 13% 5% Y3 vs. Y4: * 53% 23% 24% Y3vs. Y4:* 
2016 83% 10% 6% 58% 19% 23% 


Student Learning Objectives (SLOs) 
Teachers receive... 


2013. 57% 20% 23% Yivs.Y4:* 42% 26% 32% Y1vs. Y4:* 
Information about developing 2014 70% «13% 17% Y2vs.Y4:* 50% 16% 34% Y2vs. Y4:* 
high quality SLOs. 2015 76% 9% 14% Y3vs.Y4:* 57% 14% 29% Y3vs. Y4:* 

2016 85% 7% 7% 64% 13% 23% 

2013 50% 26% 25% Y1vs.Y4:* 32% 32% 35% Y1vs. Y4:* 


Guidance in selecting learning 2014 76% 10% 15% Y2vs.Y4:* 50% 16% 34% Y2 vs. Y4:* 


content for SLOs. 2015 79% 9% 12% Y3vs.Y4:* 56% 15% 29% Y3vs. Y4:* 
2016 87% 7% 6% 638% 13% 25% 
2013 45% | 28% 27% Yi1vs.Y4:* 27% 35% | 38% | Y1-vs. Y4:* 
Guidance in selecting 2014 63% 16% 22% Y2vs.Y4:* 37% 20% 43% Y2 vs. Y4:* 
research-based instructional 
strategies for SLOs. 2015 70% 13% 16% Y3vs. Y4: * 45% 18% 37% Y3vs. Y4:* 
2016 76% 12% 12% 52% 17% 32% 
2013 44% | 27% | 29% Yivs.Y4:* 29% 33% 37% Y1vs. Y4:* 
Recommendations for 2014. 71% 12% 18% Y2vs.Y4:* 48% 16% 36% Y2vs. Y4:* 
selecting pre/post 
assessments for SLOs. 201 5 75% 11 % 15% Y3 vs. Y4: x 53% 15% 31 % Y3 vs. Y4: x 
2016 79% 8% 13% 57% 15% 28% 


2013 42% 31% 27% Yivs.Y4:* | 28% 36% 37% | Y1 vs. Y4:* 


Valid and reliable approaches == 2014. 60% = 15% += 25% | Y2vs.Y4:* | 40% | 20% 41% Y2vs. Y4:* 
to setting growth targets for 
SLOs. 2015 65% 15% 20% Y3vs.Y4:* 47% 18% 35% Y3vs. Y4:* 


2016 77% 10% 13% 54% 17% 30% 


Note: Principals refers to principals and assistant principals; 7eachers refers to classroom teachers, non-classroom teaching staff, 
and special educators. A = Agree, a composite of strongly agree/agree. = Undecided. D= Disagree, a composite of strongly 
disagree/disagree. Y7 refers to 2013, Y2to 2014, Y3to 2015, and Y4 to 2016. * indicates statistically significant differences at the 
95% confidence level. N (principal, 2013 = 173; 2014 = 1,029; 2015 = 1,160; 2016 = 1,235); N (teacher, 2013 = 1,657; 2014 = 14,905; 
2015 = 17,344; 2016 = 20,155). 
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